Skip to main content
POST
Create Eval

Authorizations

X-Lightsage-Api-Key
string
header
required

Body

application/json
competitor_id
string
required

Tracked competitor to compare against, from GET /v2/agent-preference/competitors.

prompts
string[]
required

Buying or build questions the agent answers by choosing between your company and the competitor. CLI clients expose this as repeatable --prompt.

Required array length: 1 - 20 elements
name
string

Display name. Defaults to <your company> vs <competitor> when omitted.

Required string length: 1 - 160
agent
string[]

Runnable agents from GET /v2/agents. An empty array saves the comparison without default agents. CLI clients expose repeatable --agent.

Maximum array length: 10
persona
string

Optional perspective applied to the whole comparison.

Maximum string length: 160
search_grounding
enum<string>
default:default

Web-retrieval policy for every run: disabled, enabled, or default.

Available options:
disabled,
enabled,
default
runs
integer
default:1

Independent attempts for every prompt-agent pair.

Required range: 1 <= x <= 20
tags
string[]
Maximum array length: 20

Response

Successful Response

id
string
required

Eval ID. Pass it as eval_id to POST /v2/eval-runs to run the comparison.

name
string
required
competitor
V2AgentPreferenceCompetitor · object | null

The competitor your company is compared against. Null when the competitor is no longer tracked.

prompts
string[]
judges
string[]

Criteria applied to each response. The first one decides which company the agent recommended and cannot be changed.

agent
string[]
persona
string | null
search_grounding
enum<string>
default:default

Requested web-retrieval behavior for an eval run.

Available options:
disabled,
enabled,
default
runs
integer
default:1
Required range: 1 <= x <= 20
tags
string[]
created_at
string | null
updated_at
string | null