> ## Documentation Index > Fetch the complete documentation index at: https://lightsage.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Create Eval > Creates a head-to-head comparison between your company and one tracked competitor. Each prompt asks an agent to choose, and a fixed judge records which company it recommended. Creating the eval does not start a run; pass the returned `id` as `eval_id` to `POST /v2/eval-runs`. ## OpenAPI ````yaml /openapi.json post /v2/agent-preference/evals openapi: 3.1.0 info: title: Lightsage Public API description: Public API for Lightsage CLI and external integrations. version: 1.0.0 servers: - url: https://api.lightsage.com description: Production security: [] tags: - name: Status description: Check API availability and validate the calling API key. - name: Visibility description: >- Measure visibility, citations, competitors, and prompt performance across answer platforms. - name: Prompts description: Manage tracked prompts and topics, and inspect their stored runs. - name: Opportunities description: Review ranked opportunities computed from prompt visibility results. - name: Tasks description: Manage the persisted action queue and task lifecycle. - name: Site Audits description: Review website audit results and page-level findings. - name: Evals description: Define eval tasks, judges, subjects, and saved configurations. - name: Eval Templates description: >- Select reusable judges, targets, schedules, metadata, and runtime settings with `POST /v2/eval-templates`, then copy only those settings into prompt-specific evals with `POST /v2/evals/batch`. Templates never contain prompts and remain independent from evals created from them. - name: Eval Runs description: Start eval runs and inspect their lifecycle state. - name: Eval Results description: Review completed eval outcomes and diagnostic details. - name: Agent Preference description: >- Compare your company head-to-head with a tracked competitor. Create a comparison with `POST /v2/agent-preference/evals`, run it with `POST /v2/eval-runs`, then read which company agents recommended. - name: Agents description: List runnable coding-agent harness and model combinations. - name: Repositories description: Manage reusable Git repository workspaces. - name: Environments description: List reusable runtime environments without exposing secret values. - name: MCP Servers description: List reusable MCP server integrations without exposing credentials. - name: CLIs description: List reusable command-line interfaces without exposing setup commands. - name: Skills description: List reusable skills available to workflows and evals. paths: /v2/agent-preference/evals: post: tags: - Agent Preference summary: Create Eval description: >- Creates a head-to-head comparison between your company and one tracked competitor. Each prompt asks an agent to choose, and a fixed judge records which company it recommended. Creating the eval does not start a run; pass the returned `id` as `eval_id` to `POST /v2/eval-runs`. operationId: v2_agent_preference_evals_create requestBody: content: application/json: schema: $ref: '#/components/schemas/CreateV2AgentPreferenceEvalRequest' required: true responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/V2AgentPreferenceEval' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' security: - LightsageApiKeyAuth: [] components: schemas: CreateV2AgentPreferenceEvalRequest: properties: competitor_id: type: string title: Competitor Id description: >- Tracked competitor to compare against, from `GET /v2/agent-preference/competitors`. prompts: items: type: string type: array maxItems: 20 minItems: 1 title: Prompts description: >- Buying or build questions the agent answers by choosing between your company and the competitor. CLI clients expose this as repeatable `--prompt`. name: type: string maxLength: 160 minLength: 1 title: Name description: >- Display name. Defaults to ` vs ` when omitted. agent: items: type: string type: array maxItems: 10 title: Agent description: >- Runnable agents from `GET /v2/agents`. An empty array saves the comparison without default agents. CLI clients expose repeatable `--agent`. persona: type: string maxLength: 160 title: Persona description: Optional perspective applied to the whole comparison. search_grounding: $ref: '#/components/schemas/SearchGroundingMode' description: 'Web-retrieval policy for every run: disabled, enabled, or default.' default: default runs: type: integer maximum: 20 minimum: 1 title: Runs description: Independent attempts for every prompt-agent pair. default: 1 tags: items: type: string type: array maxItems: 20 title: Tags additionalProperties: false type: object required: - competitor_id - prompts title: CreateV2AgentPreferenceEvalRequest examples: - agent: - codex:gpt-5-4 competitor_id: 8f6f3a52-6f0d-4a4b-9f3e-0c1f4a7f2b11 prompts: - Which web search API should I use for a research agent? runs: 3 tags: - search V2AgentPreferenceEval: properties: id: type: string title: Id description: >- Eval ID. Pass it as `eval_id` to `POST /v2/eval-runs` to run the comparison. name: type: string title: Name competitor: anyOf: - $ref: '#/components/schemas/V2AgentPreferenceCompetitor' - type: 'null' description: >- The competitor your company is compared against. Null when the competitor is no longer tracked. prompts: items: type: string type: array title: Prompts judges: items: type: string type: array title: Judges description: >- Criteria applied to each response. The first one decides which company the agent recommended and cannot be changed. agent: items: type: string type: array title: Agent persona: anyOf: - type: string - type: 'null' title: Persona search_grounding: $ref: '#/components/schemas/SearchGroundingMode' default: default runs: type: integer maximum: 20 minimum: 1 title: Runs default: 1 tags: items: type: string type: array title: Tags created_at: anyOf: - type: string - type: 'null' title: Created At updated_at: anyOf: - type: string - type: 'null' title: Updated At type: object required: - id - name title: V2AgentPreferenceEval HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError SearchGroundingMode: type: string enum: - disabled - enabled - default title: SearchGroundingMode description: Requested web-retrieval behavior for an eval run. V2AgentPreferenceCompetitor: properties: id: type: string title: Id description: Competitor ID to pass as `competitor_id`. name: type: string title: Name domain: anyOf: - type: string - type: 'null' title: Domain logo_url: anyOf: - type: string - type: 'null' title: Logo Url type: object required: - id - name title: V2AgentPreferenceCompetitor ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type input: title: Input ctx: type: object title: Context type: object required: - loc - msg - type title: ValidationError securitySchemes: LightsageApiKeyAuth: type: apiKey in: header name: X-Lightsage-Api-Key ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.