> ## Documentation Index
> Fetch the complete documentation index at: https://lightsage.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Devtool Arena: the coding agent API leaderboard

> Devtool Arena is a free, public leaderboard benchmarking 85+ developer tools against AI coding agents across API, CLI, and MCP evaluation types.

Devtool Arena is a free, public leaderboard that benchmarks developer tools — APIs, SDKs, CLIs, and MCP servers — against real AI coding agents. Lightsage runs live, end-to-end evaluations and publishes the results so API teams can see exactly how their tools rank, where they fall short, and how they compare to competitors. The leaderboard is updated multiple times per week, and each row shows a timestamp so you always know how fresh the data is.

## Three leaderboard types

<CardGroup cols={3}>
  <Card title="API leaderboard" icon="code" href="https://lightsage.com/leaderboard">
    Ranks APIs evaluated directly by Claude Code using live production API keys. The primary leaderboard with the broadest coverage — 85+ tools across 17 categories.
  </Card>

  <Card title="CLI leaderboard" icon="terminal" href="https://lightsage.com/leaderboard/cli">
    Ranks developer tools by how well agents discover and use their command-line interfaces end-to-end. Covers 75 tools.
  </Card>

  <Card title="MCP leaderboard" icon="server" href="https://lightsage.com/leaderboard/mcp">
    Ranks tools with Model Context Protocol servers. Tests whether agents can use the MCP server to complete real tasks. Covers 58 tools.
  </Card>
</CardGroup>

## Who uses the leaderboard

API and developer tool teams use Devtool Arena to answer questions they previously had no data for: Does my API work when Claude Code tries to use it? How do I rank against my competitors? What does an agent actually do when it encounters my documentation? Teams monitor their score after shipping documentation changes, MCP servers, or llms.txt files to measure the impact directly.

## Categories covered

The leaderboard spans 17 categories of developer tools:

<CardGroup cols={2}>
  <Card title="Inference" icon="brain">
    LLM inference APIs — Groq, OpenRouter, Cerebras, Fireworks AI, SambaNova, and more.
  </Card>

  <Card title="Auth" icon="lock">
    Authentication and identity platforms — Clerk, Auth0, WorkOS, Descope, Stytch, Scalekit.
  </Card>

  <Card title="Payment" icon="credit-card">
    Payment processing and billing APIs — Stripe, PayPal, Razorpay, Mollie, Paddle, Chargebee, LemonSqueezy, Airwallex.
  </Card>

  <Card title="Search" icon="magnifying-glass">
    Search and web scraping APIs — Firecrawl, Jina AI, Tavily, Exa, You.com, Brave Search.
  </Card>

  <Card title="Voice STT" icon="microphone">
    Speech-to-text APIs — AssemblyAI, Deepgram, OpenAI Whisper, Rev AI, Speechmatics.
  </Card>

  <Card title="Voice TTS" icon="volume-high">
    Text-to-speech APIs — ElevenLabs, Cartesia, Rime, LMNT, Resemble AI.
  </Card>

  <Card title="Voice Telephony" icon="phone">
    Programmable voice and telephony platforms — Twilio, Plivo, Telnyx, Vonage, Sinch, Infobip.
  </Card>

  <Card title="Voice Infra" icon="tower-broadcast">
    Real-time voice infrastructure — LiveKit, Daily, Agora, Pipecat, 100ms.
  </Card>

  <Card title="Sandboxes" icon="box">
    Cloud execution environments — E2B, Daytona, Modal, Vercel, Cloudflare Workers, Sprites.
  </Card>

  <Card title="Vector Databases" icon="database">
    Vector search and embedding stores — Chroma, Pinecone, Qdrant, LanceDB, Weaviate, Zilliz Cloud.
  </Card>

  <Card title="Stablecoin" icon="coins">
    Crypto payments and stablecoin APIs — Circle, Coinbase Payments, DFNS, Fireblocks, Triple-A.
  </Card>

  <Card title="Meeting Bot" icon="video">
    Meeting recording and transcription APIs — Recall.ai, Meeting BaaS, MeetGeek, Meetstream, CueMeet.
  </Card>

  <Card title="Durable Workflow" icon="arrows-rotate">
    Workflow orchestration platforms — Temporal, Prefect, Restate, Camunda, Netflix Conductor, AWS Step Functions, Akka, Cadence.
  </Card>

  <Card title="Cloud Hosting" icon="cloud">
    Cloud deployment platforms — Vercel, Railway, Render, Netlify, DigitalOcean, Cloudflare.
  </Card>

  <Card title="Observability" icon="chart-line">
    Monitoring and metrics APIs — Datadog.
  </Card>

  <Card title="Email" icon="envelope">
    Programmatic email APIs — Resend, Agentmail.
  </Card>
</CardGroup>

## How to access the leaderboard

Visit [lightsage.com/leaderboard](https://lightsage.com/leaderboard) to view the full table. Click or tap any row to explore the detailed eval report for that tool. Use the category filters at the top to narrow the view to a specific vertical, and toggle **Skipped** to show entries where evaluation was not possible (for example, because no public API key was available).

The leaderboard displays a **Last updated** timestamp so you can see exactly when the data was refreshed.

## How to read the leaderboard table

Each row in the leaderboard represents one tool evaluated by Claude Code. Here is what each column means:

| Column        | Description                                                                                                                    |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| **Rank**      | Overall position on the leaderboard, sorted by Score descending.                                                               |
| **Tool**      | The name and logo of the evaluated tool.                                                                                       |
| **Score**     | Overall score from 0 to 100, combining Eval Score and Discovery Score.                                                         |
| **Grade**     | Letter grade (A, B, C, D) derived from the overall score.                                                                      |
| **Eval**      | Eval Score — how successfully the agent completed the assigned task end-to-end.                                                |
| **Discovery** | Discovery Score — how easily the agent found and understood the API without hand-holding.                                      |
| **Cost**      | The dollar cost in Claude API spend to run one evaluation. Lower is better.                                                    |
| **Calls**     | The number of tool calls the agent made to complete the task. Fewer calls generally indicate a simpler, better-documented API. |
| **Errors**    | The number of errors encountered during the evaluation run.                                                                    |
| **Time**      | Wall-clock time to complete the evaluation task.                                                                               |
| **C7**        | Whether the tool is indexed on Context7.                                                                                       |
| **llms**      | Whether the tool has a published `llms.txt` file.                                                                              |
| **MCP**       | Whether the tool provides an MCP server.                                                                                       |
| **SDK**       | Whether the tool offers an official SDK.                                                                                       |
| **API**       | Whether the tool exposes a REST or similar API.                                                                                |
| **Skills**    | Whether the tool has published Agent Skills.                                                                                   |
| **CLI**       | Whether the tool ships a CLI.                                                                                                  |

The checklist columns (C7 through CLI) are binary — they indicate presence or absence of each asset in the tool's publicly available resources.

<Note>
  Entries showing `—` across Score and all metrics were skipped because Lightsage could not obtain a working API key. Common reasons include requiring a paid plan before issuing credentials, requiring company verification, or being open-source / self-hosted with no hosted API available.
</Note>

## Get your API evaluated and listed

Any API or developer tool company can request an evaluation. Submit your documentation URL or GitHub repo URL at [lightsage.com/leaderboard/add](https://lightsage.com/leaderboard/add) and Lightsage will schedule a live evaluation run.

***

<CardGroup cols={3}>
  <Card title="How scoring works" icon="calculator" href="/docs/leaderboard/scoring">
    Understand how Eval Score and Discovery Score combine into your overall score, and what the grade thresholds mean.
  </Card>

  <Card title="Submit your API" icon="plus" href="/docs/leaderboard/submit-api">
    Learn what to prepare before submitting and how to maximize your score from day one.
  </Card>

  <Card title="Changelog" icon="clock-rotate-left" href="/docs/leaderboard/changelog">
    See every update to the leaderboard: new companies, new categories, and ranking changes.
  </Card>
</CardGroup>