> ## Documentation Index
> Fetch the complete documentation index at: https://lightsage.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Run docs evals from GitHub

> Use Lightsage slash commands on GitHub pull requests to run docs evals against staging documentation.

Use GitHub PR evals to test documentation changes against a staging docs URL before merging. Lightsage runs the configured custom evals for the connected docs source and stores the results in your eval history.

## Commands

| Command                | What it does                                                 |
| ---------------------- | ------------------------------------------------------------ |
| `/lightsage run evals` | Queues docs evals for the pull request.                      |
| `/lightsage status`    | Shows the latest Lightsage eval status for the pull request. |
| `/lightsage`           | Shows available Lightsage commands.                          |
| `/lightsage help`      | Shows available Lightsage commands.                          |

Put the command on the first non-empty line of a pull request comment.

## Run evals

<Steps>
  <Step title="Confirm the pull request has a staging docs URL">
    The URL can appear in the PR body, a PR comment, or a deployment attached to the pull request.
  </Step>

  <Step title="Comment on the pull request">
    Add the run command:

    ```text theme={null}
    /lightsage run evals
    ```
  </Step>

  <Step title="Wait for Lightsage to reply">
    Lightsage posts a comment when the evals are queued or when it needs more information.
  </Step>

  <Step title="Review the eval history">
    In Lightsage, open **Custom evals** and filter the history source to **PR bot**.
  </Step>
</Steps>

## Include the docs URL in the command comment

If the pull request has more than one preview URL, include the docs URL in the same comment as the command:

```text theme={null}
/lightsage run evals

Docs preview: https://example-docs-preview.vercel.app
```

Lightsage uses that URL for the eval run.

## How Lightsage finds the docs URL

Lightsage searches the pull request context for a docs URL. It checks:

* The `/lightsage run evals` comment.
* The pull request body.
* Other pull request comments.
* Deployments attached to the pull request.

If Lightsage cannot find a docs URL, it does not queue evals. It posts a comment explaining that a staging docs URL is required.

## Permissions

The comment author must have write, maintain, or admin access to the repository. This prevents drive-by comments from running evals and using credits.

The repository also needs the Lightsage GitHub App installed and connected to a Lightsage docs source.

## Check status

Use `/lightsage status` to see the latest Lightsage eval activity for the pull request:

```text theme={null}
/lightsage status
```

Lightsage replies with the latest queued or running eval status it can find for that PR.

## Troubleshooting

<AccordionGroup>
  <Accordion title="Lightsage cannot find a docs URL">
    Add the staging docs URL to the `/lightsage run evals` comment, the PR body, or a PR comment. Then run the command again.
  </Accordion>

  <Accordion title="Lightsage says the command is not authorized">
    Ask someone with write, maintain, or admin access to run the command, or update the comment author's repository permissions.
  </Accordion>

  <Accordion title="No evals are configured">
    Configure custom evals for the connected docs source in Lightsage, then run the command again.
  </Accordion>

  <Accordion title="The run used the wrong preview URL">
    Run the command again and include the intended URL directly in the command comment.
  </Accordion>
</AccordionGroup>