Devtool CLI Arena

Benchmarking developer tools evaluated by Codex Codex across auth, inference, voice, and more.

API CLI MCP

Search a tool (e.g. Stripe, Chroma)

Compare companies

Select 2-4 rows to compare scores, discoverability, cost, errors, and eval details.

Show companies whose evals were skipped (e.g. no API key provided)

Last updated Sep 7Last updated September 7, 2026 at 2:58 PM UTC

Lightsage

ClickTap any row to explore the full CLI eval.
Get your CLI graded →

Rank	Tool	Score	Grade	Task	Eval	Discovery	Cost	Calls	Errors	Time
1🥇	WorkOS	85	B		81	95	$0.67	9	0	2m 58s	View report
2🥈	LiveKit	82	B		77	93	$0.62	19	0	2m 50s	View report
3🥉	Nimble	81	B		79	87	$0.77	16	0	1m 22s	View report
3🥉	Browserbase	81	B		75	95	$0.80	23	0	2m 3s	View report
5	Cloudflare	80	B		72	100	$1.34	31	0	4m 48s	View report
6	Sprites	78	B		89	53	$0.36	9	0	46s	View report
7	E2B	77	B		85	60	$0.50	13	0	56s	View report
8	Deepgram	76	B		68	95	$0.86	11	2	2m 41s	View report
8	Tavily	76	B		77	75	$1.20	25	0	1m 51s	View report
8	Resend	76	B		66	100	$0.68	18	4	3m 1s	View report
11	Firecrawl	74	C		74	75	$0.85	25	0	3m 50s	View report
12	Stripe	73	C		65	93	$0.96	19	2	4m 3s	View report
12	Naive	73	C		75	68	$1.17	19	0	2m 29s	View report
12	Datadog	73	C		73	75	$0.33	10	1	2m 14s	View report
15	Render	72	C		61	100	$2.35	42	8	7m 8s	View report
15	Agentmail	72	C		66	86	$0.59	18	3	3m 54s	View report
15	Steel	72	C		63	93	$2.07	35	3	2m 48s	View report
18	Extend.ai	71	C		68	80	$0.82	13	3	2m 25s	View report
19	Daytona	70	C		70	70	$0.74	15	3	1m 16s	View report
20	Jina AI	69	C		69	70	$0.75	16	1	1m 40s	View report
20	Pinecone	69	C		65	78	$1.12	20	2	2m 20s	View report
20	Browser Use	69	C		64	82	$2.02	24	1	2m 55s	View report
23	ElevenLabs	68	C		61	86	$1.75	29	4	5m 19s	View report
24	Anchor Browser	66	C		63	73	$1.70	37	4	2m 46s	View report
25	AssemblyAI	64	C		62	70	$3.42	38	13	4m 41s	View report
25	Box	64	C		65	63	$1.28	13	4	3m 39s	View report
25	Modal	64	C		70	50	$0.47	17	1	1m 12s	View report
28	Temporal	61	C		66	50	$0.80	15	4	3m 19s	View report
28	Reducto	61	C		70	40	$0.81	12	1	1m 31s	View report
28	Tenki	61	C		69	43	$0.81	19	2	1m 30s	View report
31	OpenRouter	60	C		61	60	$3.20	42	8	7m 19s	View report
31	LlamaParse	60	C		69	40	$0.85	18	5	1m 55s	View report
33	Groq	59	C		61	55	$3.33	27	6	5m 26s	View report
33	Descope	59	C		61	55	$1.44	38	3	5m 32s	View report
35	Auth0	58	C		64	45	$0.86	22	1	4m 10s	View report
36	Camunda	57	C		61	50	$3.08	63	10	9m 2s	View report
37	Unstructured	56	C		64	40	$1.98	25	5	2m 44s	View report
38	Rime	52	D		62	30	$1.21	30	3	3m 40s	View report
38	Telnyx	52	D		59	35	$1.88	28	4	5m 58s	View report
38	Twilio	52	D		60	35	$1.35	25	12	6m 9s	View report
41	Vercel	48	D		36	78	$0.77	15	4	2m 6s	View report
42	Vonage	45	D		63	5	$0.83	25	9	5m 24s	View report
43	Nebius AI Cloud	39	D		28	67	$1.36	40	9	2m 14s	View report
44	Railway	29	D		2	93	$1.35	28	3	4m 49s	View report
45	Unblocked	28	D		25	35	$1.69	29	5	2m 18s	View report
46	Cartesia	21	D		6	55	$2.75	48	2	2m 41s	View report
46	Chroma	21	D		5	60	$3.26	47	9	8m 59s	View report
46	Netlify	21	D		1	68	$2.25	42	11	8m 48s	View report
49	Fireworks AI	17	D		4	50	$4.31	44	8	8m 12s	View report
50	Semgrep	13	D		6	30	$1.18	14	7	2m 34s	View report
51	CodeRabbit	12	D		5	30	$0.87	23	6	2m 55s	View report
51	Greptile	12	D		4	30	$1.19	25	3	2m 54s	View report
53	Scalekit	10	D		1	33	$2.69	33	7	6m 17s	View report
54	Qodo	9	D		1	30	$2.28	35	8	5m 4s	View report
55	Pipecat	4	D		1	13	$2.73	41	14	7m 50s	View report
56	Zilliz Cloud Missing CLI auth env vars: ZILLIZ_API_KEY	—	—	—	—	—	—	—	—	—	View report
56	DigitalOcean Signup requires a credit card	—	—	—	—	—	—	—	—	—	View report
58	Nylas Missing CLI auth env vars: NYLAS_GRANT_ID	—	—	—	—	—	—	—	—	—	View report
59	Weaviate Missing CLI auth env vars: WEAVIATE_API_KEY, WEAVIATE_URL	—	—	—	—	—	—	—	—	—	View report
60	Sourcegraph Missing CLI auth env vars: SRC_ENDPOINT, SRC_ACCESS_TOKEN	—	—	—	—	—	—	—	—	—	View report