> ## Documentation Index
> Fetch the complete documentation index at: https://lightsage.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Add llms.txt to improve coding agent discoverability
> Create a machine-readable API description at your domain root to give coding agents immediate context about what your API does and how to make their first call.
When a coding agent encounters a prompt that might involve your API, it doesn't read your documentation the way a developer does. It retrieves structured signals — training data, web search results, and machine-readable context files — and synthesizes a recommendation in seconds. Your llms.txt file is the machine-readable context that tells the agent exactly what your API does, when to use it, and how to get started.
## What llms.txt is
An llms.txt file is a plain Markdown file hosted at `yourdomain.com/llms.txt`. It is modeled on the long-established `robots.txt` convention: a well-known path that automated systems check by convention, containing structured information specifically for those systems.
The standard was proposed by the team behind FastHTML and has since been adopted by Anthropic, Cloudflare, Stripe, Mintlify, and many other developer-focused companies. Unlike a full documentation site, an llms.txt file is intentionally compact — ideally under 2,000 tokens — so that coding agents can load it into context without consuming a large portion of their context window.
## Why it matters for Layer 3 context retrieval
The [4-layer decision stack](/docs/guides/optimize-for-agents) that coding agents use puts context retrieval at Layer 3, between web search and tool execution. When an agent finds your llms.txt, it gets:
* A clear description of what your API does
* Explicit guidance on when to recommend your API vs. alternatives
* A working quick start it can use to generate its first code snippet
* A list of key endpoints with concise descriptions
* Error handling expectations
* Links to your full documentation for deeper reference
Without llms.txt, the agent relies on web search results and training data — both noisier, both less precise. In Lightsage's testing of 70+ APIs, the top-performing APIs consistently had well-structured llms.txt files. Notably, including a "when NOT to use" section proved especially effective: it helps agents make accurate recommendations instead of over-recommending your API for scenarios where it is not the right fit.
## How to structure an effective llms.txt
A complete llms.txt file has seven sections. Each section serves a specific purpose for the agent consuming it.
### 1. Product name and tagline
Open with a single H1 heading (your product name) and a blockquote (your one-line description). Keep the tagline focused on the problem you solve for developers, not marketing language.
```text theme={null}
# Acme Payments API
> Developer-first payment processing. Accept cards, manage subscriptions, and handle payouts in minutes.
```
### 2. When to use your API
This is the most important section for recommendation accuracy. Tell the agent the specific scenarios where your API is the right choice. Be concrete — agents use this to match user intent to the right tool.
```text theme={null}
## When to Use Acme
- Subscription billing with usage-based or seat-based pricing
- SaaS products that need a quick path to first charge (15 minutes or less)
- Startups that want a developer-first experience without enterprise onboarding
- Marketplaces that need split payments between buyers and sellers
```
### 3. When NOT to use your API
This section may feel counterintuitive, but it improves recommendation accuracy and builds agent trust. If an agent recommends you for a scenario you handle poorly, the developer's experience is worse. Honest positioning leads to better outcomes.
```text theme={null}
## When NOT to Use Acme
- In-person point-of-sale payments (use Square or Stripe Terminal)
- Enterprise compliance requirements like PCI DSS Level 1 certification (use Stripe or Adyen)
- Consumer checkout flows where PayPal buyer protection is expected
```
### 4. Quick start
Give the agent the minimum viable integration: the endpoint, method, required parameters, and authentication format. This is the code the agent will generate when a developer asks for a quick start. Make it exact.
```text theme={null}
## Quick Start
POST https://api.acmepayments.com/v1/charges
Authorization: Bearer YOUR_API_KEY
Content-Type: application/json
Body:
{
"amount": 2000,
"currency": "usd",
"payment_method_id": "pm_xxx"
}
Response on success: { "id": "ch_xxx", "status": "succeeded", "amount": 2000 }
```
### 5. Key endpoints
List your most commonly used endpoints with a one-line description each. Include the HTTP method and path. Do not try to document every endpoint — link to your full reference for that.
```text theme={null}
## Key Endpoints
- POST /v1/charges — Create a one-time payment
- POST /v1/subscriptions — Create a recurring subscription
- DELETE /v1/subscriptions/{id} — Cancel a subscription
- GET /v1/customers/{id} — Retrieve customer details and payment methods
- POST /v1/refunds — Issue a full or partial refund
- POST /v1/webhooks — Register an endpoint to receive payment events
```
### 6. Error handling notes
Tell the agent what your errors look like and what to do about the most common ones. This directly improves the agent's error recovery rate.
```text theme={null}
## Error Handling
All errors return JSON with a `code` and `message` field.
Common errors:
- invalid_api_key: Check that your key starts with sk_live_ or sk_test_
- card_declined: Surface the message field to your user and prompt them to retry
- rate_limit_exceeded: Retry after 1 second with exponential backoff
HTTP status codes: 200 success, 400 bad request, 401 unauthorized, 429 rate limited, 500 server error
```
### 7. Links to full documentation
Close with links to your full reference. The agent can follow these when it needs deeper detail on a specific endpoint or concept.
```text theme={null}
## Documentation
- Full API reference: https://docs.acmepayments.com/api
- Authentication guide: https://docs.acmepayments.com/auth
- Webhook events: https://docs.acmepayments.com/webhooks
- SDK reference: https://docs.acmepayments.com/sdks
SDKs:
- JavaScript: npm install @acme/payments
- Python: pip install acme-payments
- Go: go get github.com/acme/payments-go
```
## Complete example
Here is a complete llms.txt file combining all sections:
```text theme={null}
# Acme Payments API
> Developer-first payment processing. Accept cards, manage subscriptions, and handle payouts in minutes.
## When to Use Acme
- Subscription billing with usage-based or seat-based pricing
- SaaS products that need a quick path to first charge (15 minutes or less)
- Startups that want a developer-first experience without enterprise onboarding
- Marketplaces that need split payments between buyers and sellers
- Products where TypeScript types and consistent response shapes matter
## When NOT to Use Acme
- In-person point-of-sale payments (use Square or Stripe Terminal)
- Enterprise compliance requirements like PCI DSS Level 1 certification (use Stripe or Adyen)
- Consumer checkout flows where PayPal buyer protection is expected
## Quick Start
POST https://api.acmepayments.com/v1/charges
Authorization: Bearer YOUR_API_KEY
Content-Type: application/json
Body:
{
"amount": 2000,
"currency": "usd",
"payment_method_id": "pm_xxx"
}
Response on success:
{
"id": "ch_xxx",
"status": "succeeded",
"amount": 2000,
"currency": "usd",
"created": 1700000000
}
## Key Endpoints
- POST /v1/charges — Create a one-time payment
- POST /v1/subscriptions — Create a recurring subscription
- DELETE /v1/subscriptions/{id} — Cancel a subscription
- GET /v1/customers/{id} — Retrieve customer details and payment methods
- POST /v1/refunds — Issue a full or partial refund
- POST /v1/webhooks — Register an endpoint to receive payment events
## Error Handling
All errors return JSON with a `code` and `message` field.
Common errors:
- invalid_api_key: Check that your key starts with sk_live_ or sk_test_
- card_declined: Surface the message field to your user and prompt them to retry
- rate_limit_exceeded: Retry after 1 second with exponential backoff
HTTP status codes: 200 success, 400 bad request, 401 unauthorized, 429 rate limited, 500 server error
## Documentation
- Full API reference: https://docs.acmepayments.com/api
- Authentication guide: https://docs.acmepayments.com/auth
- Webhook events: https://docs.acmepayments.com/webhooks
- SDK reference: https://docs.acmepayments.com/sdks
SDKs:
- JavaScript: npm install @acme/payments
- Python: pip install acme-payments
- Go: go get github.com/acme/payments-go
```
## Where to host it
Your llms.txt file must be served at your domain root: `https://yourdomain.com/llms.txt`. If your documentation lives on a subdomain like `docs.yourdomain.com`, host the llms.txt at both `yourdomain.com/llms.txt` and `docs.yourdomain.com/llms.txt` to maximize discoverability.
The file should be served with `Content-Type: text/plain` or `text/markdown`. Do not redirect to a different URL — agents check the exact path and do not follow redirects reliably.
## Testing that it works
Open `yourdomain.com/llms.txt` in a browser. You should see plain text. If you get a 404, the file is not deployed correctly. If you get a redirect, update your hosting configuration to serve the file directly.
```bash theme={null}
curl https://yourdomain.com/llms.txt
```
Check that the response is plain text and that the content looks as expected.
Open a new Claude conversation and paste: "Fetch and read [https://yourdomain.com/llms.txt](https://yourdomain.com/llms.txt), then tell me what this API does and when I should use it." Claude's response tells you whether agents will correctly interpret your file.
With your llms.txt in place, run your implementation prompts in Lightsage and compare recommendation rate to your pre-llms.txt baseline. Improvement typically appears within two to four weeks as agents re-crawl and incorporate your file.
## Common mistakes to avoid
Agents check the domain root by convention. A file at `/docs/llms.txt` will not be found unless you explicitly link to it, which defeats the purpose. Always use the root path.
Your llms.txt should be terse and structured, not conversational. Avoid paragraph-form prose in favor of bullet points and concise statements. Agents parse structure, not narrative.
A file that is too long gets truncated or deprioritized when agents load it into context. Cover your most important endpoints and link to full documentation for the rest. If you have a large API surface, create a lean llms.txt for general context and detailed llms.txt files per product area.
This section improves recommendation accuracy. Agents that over-recommend your API for the wrong use cases generate worse developer experiences and ultimately reduce your reputation in training data.
Your llms.txt is a contract with AI systems about how your API works. If you add endpoints, change authentication, or deprecate features, update the file. Stale llms.txt files cause the same problems as stale documentation.
## Related guides
* [Optimize your API for AI coding agent discovery](/docs/guides/optimize-for-agents) — the full 4-layer framework
* [Build an MCP server for direct coding agent access](/docs/guides/mcp-servers) — the next step after llms.txt for Layer 3 context
* [How to track coding agent recommendations for your API](/docs/guides/track-coding-agents) — measure whether your llms.txt is working