> ## Documentation Index
> Fetch the complete documentation index at: https://lightsage.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom eval runs

> Run custom evals, list executions, and inspect live run state and traces from the Lightsage CLI.

Custom eval runs are first-class Lightsage CLI workflows. Start configured custom evals, list active and completed executions, poll one run, and retrieve its trace with the same `run_id`.

## Commands

| Command                                 | Description                                       |
| --------------------------------------- | ------------------------------------------------- |
| `lightsage run evals`                   | Start configured custom evals.                    |
| `lightsage list evals`                  | List active and completed custom eval executions. |
| `lightsage get run --run-id <run_id>`   | Get lifecycle and result state for one execution. |
| `lightsage get trace --run-id <run_id>` | Get a completed trace or poll trace availability. |

## Run custom evals

Run selected custom eval definitions:

```bash theme={null}
lightsage run evals \
  --eval-ids '["custom_eval_123"]'
```

If you omit `--eval-ids`, Lightsage uses the custom eval set configured for your organization. You can override configured coding-agent targets for a one-off run:

```bash theme={null}
lightsage run evals \
  --eval-ids '["custom_eval_123"]' \
  --targets '[{"target_type":"agent","execution_platform":"codex","model_id":"gpt-5","enabled":true}]'
```

The response includes a parent `batch_id`, a `dispatch_id`, and the individual executions in `runs`. Use each child `run_id` with `lightsage get run` and `lightsage get trace`.

## List eval executions

```bash theme={null}
lightsage list evals --limit 50
```

Filter active executions or one custom eval definition:

```bash theme={null}
lightsage list evals \
  --status running \
  --eval-id custom_eval_123
```

Each row includes lifecycle status, stage, elapsed time, trace availability, and a full `dashboard_url` containing your organization ID and the selected run ID.

## Get a run

```bash theme={null}
lightsage get run --run-id job_123
```

While the execution is active, poll this command to read the latest `status`, `stage`, and `elapsed_seconds`. When the result is persisted, `result_id` is populated and `trace_available` becomes `true`.

Statuses include `pending`, `running`, `completed`, `failed`, `cancelled`, and `interrupted`.

## Get a trace

```bash theme={null}
lightsage get trace --run-id job_123
```

For an active execution, the response returns `trace_available: false` with the current status, stage, and elapsed time. After completion, the response includes the prompt, response, tool calls, conversation turns, execution output, generated files, and exit code when available.