Runs

Runs are individual API Performance eval results. Use runs commands to inspect recent results and debug specific failures. Run responses include a source object when the run belongs to a configured API Performance source. The source.type identifies the surface Lightsage evaluated, such as rest_api, python_sdk, node_sdk, or cli. Source metadata is limited to public setup fields, such as package names or install commands. Secret environment variable values are never returned.

Commands

Command	Description
`lightsage api-performance runs list`	List recent eval runs.
`lightsage api-performance runs retrieve --run-id <run_id>`	Retrieve one eval run.
`lightsage api-performance diagnose`	Diagnose failures across a recent time window.

List recent runs

lightsage api-performance runs list --limit 50

Filter by time window:

lightsage api-performance runs list --since 7d

Example response:

{
  "company": {
    "name": "Firecrawl",
    "domain": "firecrawl.dev"
  },
  "data": [
    {
      "id": "run_123",
      "endpoint": {
        "id": "endpoint_123",
        "method": "GET",
        "path": "/crawl"
      },
      "source": {
        "id": "int_python",
        "type": "python_sdk",
        "label": "Python SDK",
        "metadata": {
          "language": "python",
          "package_name": "firecrawl"
        }
      },
      "eval_type": "integration",
      "model": "claude-sonnet-4.5",
      "passed": true,
      "score": 1.0,
      "run_at": "2026-05-23T00:00:00Z"
    }
  ],
  "meta": {
    "limit": 50,
    "count": 1
  }
}

Retrieve a run

lightsage api-performance runs retrieve --run-id run_123

The retrieve response includes run detail, prompt text, model output, execution logs, tool calls, and raw debugging fields when available.

{
  "data": {
    "id": "run_123",
    "endpoint": {
      "id": "endpoint_123",
      "method": "GET",
      "path": "/crawl"
    },
    "source": {
      "id": "int_rest",
      "type": "rest_api",
      "label": "REST API",
      "metadata": {
        "api_base_url": "https://api.example.com"
      }
    },
    "eval_type": "integration",
    "model": "claude-sonnet-4.5",
    "passed": false,
    "score": 0.42,
    "failure_reasons": ["The request used the wrong endpoint."],
    "prompt_sent": "Use the API to crawl a page.",
    "response_text": "The agent response text.",
    "tool_calls": [],
    "exit_code": 0
  }
}

Diagnose failures

Diagnose recent runs and return structured failure patterns:

lightsage api-performance diagnose --since 20d --format-param markdown

Include example failed runs:

lightsage api-performance diagnose \
  --since 7d \
  --include-examples \
  --max-examples 10

Useful flags:

Flag	Description
`--since`	Lookback window such as `20d`, `72h`, or an ISO timestamp.
`--format-param`	Diagnosis report format. Use `json`, `markdown`, or `md`.
`--include-examples`	Include example failed runs.
`--max-examples`	Maximum failed run examples to include.

CLI Reference

Prompts

API Performance

Commands

List recent runs

Retrieve a run

Diagnose failures

​Commands

​List recent runs

​Retrieve a run

​Diagnose failures

Commands

List recent runs

Retrieve a run

Diagnose failures