> ## Documentation Index
> Fetch the complete documentation index at: https://lightsage.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Devtool Arena changelog: scores, companies, categories

> A running record of all notable changes to the Devtool Arena leaderboard: new companies, new categories, new leaderboard types, and ranking updates.

This page documents all notable changes to the Devtool Arena leaderboard, including new companies added, new categories launched, new leaderboard types, and significant ranking updates.

<Note>
  Leaderboard data is updated multiple times per week. Subscribe to the mailing list at [lightsage.com/research](https://lightsage.com/research) for updates delivered directly to your inbox.
</Note>

***

## May 12, 2026

Updated the **Claude Code API leaderboard** with the latest evaluation runs across **85 companies**. Firecrawl holds #1 with a score of 86; OpenRouter and Jina AI are tied at #2 with 84.

### Top 10 Overall (API leaderboard)

| Rank | Company    | Category         | Score | Grade |
| ---- | ---------- | ---------------- | ----- | ----- |
| 1    | Firecrawl  | Search           | 86    | B     |
| 2    | OpenRouter | Inference        | 84    | B     |
| 2    | Jina AI    | Search           | 84    | B     |
| 4    | Datadog    | Observability    | 83    | B     |
| 5    | Prefect    | Durable Workflow | 82    | B     |
| 5    | Stripe     | Payment          | 82    | B     |
| 7    | PayPal     | Payment          | 81    | B     |
| 7    | Tavily     | Search           | 81    | B     |
| 9    | AssemblyAI | Voice STT        | 80    | B     |
| 9    | You.com    | Search           | 80    | B     |

***

## May 9, 2026

Updated the **Codex MCP leaderboard** with gpt-5.5 runs across **68 companies**. Jina AI holds #1 with 85; Stripe enters at #2 with 83. Added **8 new companies** including Telnyx, Twilio, Resemble AI, Pipecat, and Fireworks AI.

### New Companies

| Company      | Category        | Description                                                 | Score | Grade | Overall Rank | Category Rank |
| ------------ | --------------- | ----------------------------------------------------------- | ----- | ----- | ------------ | ------------- |
| Telnyx       | Voice Telephony | Programmable voice, messaging, and SIP trunking API         | 66%   | C     | #13 of 68    | #1 of 7       |
| Plivo        | Voice Telephony | Voice and SMS API platform for building communications apps | 59%   | C     | #19 of 68    | #2 of 7       |
| Twilio       | Voice Telephony | Cloud communications platform for voice, SMS, and video     | 57%   | C     | #22 of 68    | #3 of 7       |
| Resemble AI  | Voice TTS       | AI voice cloning and text-to-speech synthesis API           | 47%   | D     | #27 of 68    | #3 of 3       |
| Sinch        | Voice Telephony | Cloud communications API for voice, SMS, and messaging      | 10%   | D     | #52 of 68    | #4 of 7       |
| Vonage       | Voice Telephony | Programmable communications API for voice and messaging     | 10%   | D     | #52 of 68    | #4 of 7       |
| Pipecat      | Voice Infra     | Open-source framework for real-time voice AI pipelines      | 4%    | D     | #66 of 68    | #2 of 2       |
| Fireworks AI | Inference       | Fast inference API for open-source LLMs and image models    | 4%    | D     | #66 of 68    | #5 of 5       |

### Top 10 Overall (Codex MCP leaderboard)

| Rank | Company    | Category         | Score | Grade |
| ---- | ---------- | ---------------- | ----- | ----- |
| 1    | Jina AI    | Search           | 85%   | B     |
| 2    | Stripe     | Payment          | 83%   | B     |
| 3    | ElevenLabs | Voice TTS        | 81%   | B     |
| 3    | Tavily     | Search           | 81%   | B     |
| 5    | Scrapfly   | Webscraping      | 80%   | B     |
| 6    | Firecrawl  | Search           | 78%   | B     |
| 6    | Render     | Cloud Hosting    | 78%   | B     |
| 8    | Descope    | Auth             | 77%   | B     |
| 9    | Agentmail  | Email            | 74%   | C     |
| 10   | Qdrant     | Vector Databases | 72%   | C     |

***

## May 8, 2026

Launched the **Codex MCP leaderboard** with **53 companies** scored. Added **Voice Telephony** as a new category with Bandwidth and Infobip. Jina AI leads at #1 with a score of 81; Firecrawl and Agentmail round out the top 3.

### New Categories

* **Voice Telephony** — Telephony and programmable voice call APIs

### New Companies

| Company           | Category        | Description                                                        | Score | Grade | Overall Rank | Category Rank |
| ----------------- | --------------- | ------------------------------------------------------------------ | ----- | ----- | ------------ | ------------- |
| Nylas             | Email           | Email and calendar API for reading, sending, and managing messages | 19%   | D     | #9 of 53     | #3 of 3       |
| Coinbase Payments | Stablecoin      | Stablecoin and crypto payments API from Coinbase                   | 10%   | D     | #37 of 53    | #1 of 2       |
| Fireblocks        | Stablecoin      | Digital asset custody and stablecoin transfer infrastructure       | 10%   | D     | #39 of 53    | #2 of 2       |
| Bandwidth         | Voice Telephony | Programmable voice and messaging carrier API                       | 9%    | D     | #43 of 53    | #1 of 2       |
| Infobip           | Voice Telephony | Omnichannel communications platform with voice call and SMS APIs   | 4%    | D     | #53 of 53    | #2 of 2       |

### Top 10 Overall (Codex MCP leaderboard, launch)

| Rank | Company   | Category         | Score | Grade |
| ---- | --------- | ---------------- | ----- | ----- |
| 1    | Jina AI   | Search           | 81%   | B     |
| 2    | Firecrawl | Search           | 71%   | C     |
| 3    | Agentmail | Email            | 68%   | C     |
| 4    | Resend    | Email            | 65%   | C     |
| 5    | Clerk     | Auth             | 63%   | C     |
| 6    | Cartesia  | Voice TTS        | 61%   | C     |
| 7    | Chroma    | Vector Databases | 48%   | D     |
| 8    | Cerebras  | Inference        | 34%   | D     |
| 9    | Nylas     | Email            | 19%   | D     |
| 10   | Qdrant    | Vector Databases | 13%   | D     |

***

## April 30, 2026

Updated the **Claude Code API**, **Claude Code CLI**, **Codex API**, and **Codex CLI** leaderboards. This snapshot reflects the latest completed runs with **108 companies** listed and **954 completed evaluations** across all four leaderboards.

### Top 10 Overall (all leaderboards, best score per company)

| Rank | Company    | Category      | Score | Grade |
| ---- | ---------- | ------------- | ----- | ----- |
| 1    | ElevenLabs | Voice TTS     | 92%   | A     |
| 1    | Firecrawl  | Search        | 92%   | A     |
| 3    | Auth0      | Auth          | 91%   | A     |
| 4    | Tavily     | Search        | 90%   | A     |
| 5    | LiveKit    | Voice Infra   | 89%   | B     |
| 6    | Datadog    | Observability | 88%   | B     |
| 7    | Stripe     | Payment       | 86%   | B     |
| 8    | Jina AI    | Search        | 85%   | B     |
| 9    | OpenRouter | Inference     | 84%   | B     |
| 10   | Deepgram   | Voice STT     | 83%   | B     |

***

## April 29, 2026

Updated the **Codex CLI leaderboard**. This snapshot reflects the latest completed Codex CLI runs with **88 companies** listed and **216 completed Codex CLI evaluations**.

### Top 10 Overall (Codex CLI leaderboard)

| Rank | Company    | Category  | Score | Grade |
| ---- | ---------- | --------- | ----- | ----- |
| 1    | Firecrawl  | Search    | 86%   | B     |
| 2    | Jina AI    | Search    | 85%   | B     |
| 3    | ElevenLabs | Voice TTS | 84%   | B     |
| 3    | Tavily     | Search    | 84%   | B     |
| 5    | E2B        | Sandboxes | 82%   | B     |
| 5    | Resend     | Email     | 82%   | B     |
| 7    | Stripe     | Payment   | 80%   | B     |
| 8    | AssemblyAI | Voice STT | 79%   | B     |
| 8    | Mollie     | Payment   | 79%   | B     |
| 8    | WorkOS     | Auth      | 79%   | B     |

***

## April 26, 2026

Added the **Codex CLI leaderboard**. This snapshot reflects the initial completed Codex CLI end-to-end runs with **74 companies** listed and **33 completed Codex CLI evaluations**.

### Top 10 Overall (Codex CLI leaderboard, launch)

| Rank | Company   | Category      | Score | Grade |
| ---- | --------- | ------------- | ----- | ----- |
| 1    | LiveKit   | Voice Infra   | 89%   | B     |
| 2    | Firecrawl | Search        | 86%   | B     |
| 3    | Jina AI   | Search        | 85%   | B     |
| 4    | Tavily    | Search        | 84%   | B     |
| 5    | E2B       | Sandboxes     | 82%   | B     |
| 5    | Resend    | Email         | 82%   | B     |
| 7    | Stripe    | Payment       | 80%   | B     |
| 8    | WorkOS    | Auth          | 79%   | B     |
| 9    | Datadog   | Observability | 77%   | B     |
| 10   | Sprites   | Sandboxes     | 75%   | B     |

***

## April 24, 2026

Added the **Codex API leaderboard**. This snapshot reflects the initial completed Codex API runs with **74 companies** evaluated.

### Top 10 Overall (Codex API leaderboard, launch)

| Rank | Company    | Category      | Score | Grade |
| ---- | ---------- | ------------- | ----- | ----- |
| 1    | ElevenLabs | Voice TTS     | 92%   | A     |
| 1    | Firecrawl  | Search        | 92%   | A     |
| 3    | Tavily     | Search        | 90%   | A     |
| 4    | Datadog    | Observability | 88%   | B     |
| 5    | Stripe     | Payment       | 86%   | B     |
| 6    | OpenRouter | Inference     | 84%   | B     |
| 7    | Deepgram   | Voice STT     | 83%   | B     |
| 8    | Cartesia   | Voice TTS     | 79%   | B     |
| 8    | Daytona    | Sandboxes     | 79%   | B     |
| 8    | Mollie     | Payment       | 79%   | B     |

***

## April 15, 2026

Added the **MCP leaderboard**. This snapshot reflects the initial completed MCP runs with **74 companies** on the leaderboard and **50 companies** tracked with MCP metadata in the registry.

### Top 10 Overall (MCP leaderboard, launch)

| Rank | Company    | Category         | Score | Grade |
| ---- | ---------- | ---------------- | ----- | ----- |
| 1    | Chroma     | Vector Databases | 78%   | B     |
| 2    | Agentmail  | Email            | 77%   | B     |
| 2    | Qdrant     | Vector Databases | 77%   | B     |
| 2    | Tavily     | Search           | 77%   | B     |
| 5    | Firecrawl  | Search           | 76%   | B     |
| 5    | Jina AI    | Search           | 76%   | B     |
| 7    | ElevenLabs | Voice TTS        | 75%   | B     |
| 7    | MeetGeek   | Meeting Bot      | 75%   | B     |
| 9    | Deepgram   | Voice STT        | 74%   | C     |
| 9    | Descope    | Auth             | 74%   | C     |

***

## April 9, 2026

Added the **Claude Code CLI leaderboard**. This snapshot reflects the initial completed Claude Code CLI end-to-end runs with **74 companies** on the leaderboard and **35 companies** tracked with CLI metadata in the registry.

### Top 10 Overall (Claude Code CLI leaderboard, launch)

| Rank | Company   | Category    | Score | Grade |
| ---- | --------- | ----------- | ----- | ----- |
| 1    | LiveKit   | Voice Infra | 83%   | B     |
| 2    | Firecrawl | Search      | 80%   | B     |
| 3    | Jina AI   | Search      | 79%   | B     |
| 3    | Resend    | Email       | 79%   | B     |
| 5    | Daytona   | Sandboxes   | 78%   | B     |
| 5    | Tavily    | Search      | 78%   | B     |
| 7    | Vercel    | Sandboxes   | 77%   | B     |
| 8    | Sprites   | Sandboxes   | 74%   | C     |
| 8    | Stripe    | Payment     | 74%   | C     |
| 8    | WorkOS    | Auth        | 74%   | C     |

***

## April 5, 2026

Added 2 new categories to cover APIs that did not fit existing buckets, along with Datadog and Agentmail as the first entries in each.

### New Categories

* **Observability** — Monitoring, metrics, and event tracking APIs
* **Email** — Programmatic email inbox and messaging APIs

### New Companies

| Company   | Category      | Description                                                  | Score | Grade | Overall Rank | Category Rank |
| --------- | ------------- | ------------------------------------------------------------ | ----- | ----- | ------------ | ------------- |
| Datadog   | Observability | Python SDK for submitting custom metrics and events          | 83%   | B     | #3 of 72     | #1 of 1       |
| Agentmail | Email         | Programmatic email inboxes, send/receive, and AI agent email | 75%   | B     | #17 of 72    | #1 of 1       |

### Top 10 Overall (Claude Code API leaderboard)

| Rank | Company    | Category         | Score | Grade |
| ---- | ---------- | ---------------- | ----- | ----- |
| 1    | Firecrawl  | Search           | 86%   | B     |
| 2    | Jina AI    | Search           | 84%   | B     |
| 3    | Datadog    | Observability    | 83%   | B     |
| 4    | Auth0      | Auth             | 82%   | B     |
| 4    | Prefect    | Durable Workflow | 82%   | B     |
| 4    | Stripe     | Payment          | 82%   | B     |
| 7    | Tavily     | Search           | 81%   | B     |
| 7    | PayPal     | Payment          | 81%   | B     |
| 9    | You.com    | Search           | 80%   | B     |
| 9    | ElevenLabs | Voice TTS        | 80%   | B     |

***

## March 2026

Launched the Devtool Arena leaderboard with **72 API companies** scored across 13 initial categories.

### Initial Launch Categories and Companies

<AccordionGroup>
  <Accordion title="Inference (8 companies)">
    Groq, TogetherAI, Perplexity, OpenRouter, Cerebras, Fireworks AI, DeepInfra, SambaNova
  </Accordion>

  <Accordion title="Auth (6 companies)">
    Clerk, Auth0, Stytch, Scalekit, WorkOS, Descope
  </Accordion>

  <Accordion title="Payment (10 companies)">
    Stripe, PayPal, Square, Adyen, Razorpay, Mollie, Paddle, Chargebee, LemonSqueezy, Airwallex
  </Accordion>

  <Accordion title="Search (6 companies)">
    Exa, Tavily, You.com, Brave Search, Firecrawl, Jina AI
  </Accordion>

  <Accordion title="Voice STT (3 companies)">
    Deepgram, AssemblyAI, OpenAI Whisper
  </Accordion>

  <Accordion title="Voice TTS (2 companies)">
    ElevenLabs, Rime
  </Accordion>

  <Accordion title="Voice Infra (1 company)">
    LiveKit
  </Accordion>

  <Accordion title="Sandboxes (6 companies)">
    Vercel, Cloudflare Workers, E2B, Daytona, Modal, Fly.io
  </Accordion>

  <Accordion title="Vector Databases (6 companies)">
    Chroma, Pinecone, Weaviate, Qdrant, LanceDB, Zilliz Cloud
  </Accordion>

  <Accordion title="Stablecoin (6 companies)">
    Stripe (Crypto), Coinbase Payments, Circle, Fireblocks, DFNS, Triple-A
  </Accordion>

  <Accordion title="Meeting Bot (5 companies)">
    Recall.ai, Meeting BaaS, MeetGeek, Meetstream, CueMeet
  </Accordion>

  <Accordion title="Durable Workflow (7 companies)">
    Temporal, Prefect, Cadence, Camunda, Netflix Conductor, AWS Step Functions, Akka
  </Accordion>

  <Accordion title="Cloud Hosting (6 companies)">
    Vercel, Cloudflare, Netlify, Railway, Render, DigitalOcean
  </Accordion>
</AccordionGroup>