We raised $4M to scale our Agent-Led Growth Platform:Read more
Lightsage

Best MCP Server Testing Tools in 2026

Jun Liang LeeJun Liang Lee11 min read

Direct answer

For MCP server testing, use the MCP Inspector to check the protocol, MCPJam or MCPLab to check tool choice with models, and Lightsage to see whether real AI coding agents such as Claude Code and Codex finish a developer's task with your server installed.


How we ranked MCP server testing tools

An MCP server is a program that exposes tools, resources and prompts to AI clients over the Model Context Protocol. Testing one means answering three different questions, and most tools answer only one or two.

  1. Protocol and schema. Does the server connect, list its tools and reject bad inputs? Scalar's guide to testing MCP servers calls this ordinary software testing.
  2. Tool choice. Given a realistic request, does a model pick the right tool with the right arguments? Scalar calls this the new part, and the one that decides whether users find the server useful.
  3. End to end. Can a real AI coding agent finish a developer's task with the server installed, including the install, the auth and the recovery from errors?

We ranked by how far up those levels a tool tests with real agents, using each vendor's own pages as of 1 November 2026. Ties go to the tool that runs more named agents. Where a page is silent, we write "doesn't document".

This is a different job from LLM-app evals. Braintrust, LangSmith and DeepEval test an AI application you built. An MCP server is the product being tested here, and the AI clients are other people's. Our broader list of agent testing tools covers both kinds, plus API and CLI testing.

Why the end-to-end level matters on Devtool Arena

Devtool Arena is our free public leaderboard. Its eval score runs a real integration task end to end, and its discovery score checks that assets such as llms.txt, MCP docs and typed SDKs are public. As of 1 November 2026, the Claude Code MCP board lists 85 servers and 64 have an eval score. The Codex MCP board lists 84 and 65 have one.

On Claude Code, the 39 scored servers with a discovery score of 70 or more averaged 68.9 on the task. The 25 under 70 averaged 31.3. On Codex, the split was 26 servers at 67.7 against 39 at 45.5. Both boards show a gap, and it is wider on Claude Code. We think it shows that an agent has to find and install a server before any tool-choice test matters.

The top of each board shows what a passing server looks like. Qdrant scored 90 (A) on Claude Code with 6 tool calls and 0 errors, and 93 (A) on Codex with the same 6 calls. ElevenLabs scored 89 (B) on both boards. Only 23 servers reached 80 or more on Claude Code, and 14 on Codex. A protocol check can't tell you which side of that line a server sits on.

MCP server testing tools at a glance

ToolBest forLevels testedAgentsWhere it falls shortPrice
Lightsage MCP EvalsWhether real coding agents finish a task with your server2 and 312 coding agentsEach run takes minutesFrom $100/month; 7-day trial
Devtool Arena MCP boardsA free baseline against published servers3Claude Code, CodexScores listed servers, not your private buildFree
MCPJamTool-choice evals in CI, plus OAuth debugging1 and 216 emulated client configurationsEmulates each host with a modelFree; Team $30/seat/month billed annually
MCPLabScripted scenarios across several models2Models through their APIsTests models, not coding agentsNot listed on its page
PromptfooA free harness you grade yourself1 and 2Any model providerYou write the tasks, graders and sandboxesCommunity free; Enterprise custom
MCP InspectorDebugging protocol, tools and OAuth by hand1NoneNo model in the loopFree to run with npx
Microsoft ScopeSelf-hosted, repeatable agent experiments3GitHub Copilot, Claude Code; Codex and OpenCode plannedResearch preview; you host it; two agents todayOpen source (MIT)

1. Lightsage MCP Evals: real coding agents, in sandboxes

Lightsage runs real coding agents against your MCP server in sandboxes. You write the task, such as "set up the server and list my five most recent records", and each agent works through it. The result is pass or fail, a full trace of every tool call, the errors, and per-agent and per-tool scores. It supports 12 coding agent harnesses, including Claude Code, Codex, Cursor, GitHub Copilot, Grok Code and Muse Code, and you choose the model each one runs, so you can test the setups your users actually have.

It tests level 3 directly, and level 2 as a by-product: the trace shows which tool each agent called first and with what arguments. Devtool Arena is the public version of the same runs, so you can compare your result with other servers.

The limitation is speed. Agent runs take minutes, not seconds: the 20 highest-scoring runs on the Claude Code MCP board took between 51 seconds and 7 minutes 20 seconds. That suits a release check, and it is slower than a unit test on every commit.

Price: Free covers 10 ChatGPT prompts a week, and coding-agent evals need Scale, from $100/month, with every agent included in every tier. Runs use usage-based credits. A 7-day trial includes 1,000 credits and every agent, with no card needed (pricing).

Best for: teams that need to know whether Claude Code, Codex and others can finish a real task with their MCP server, and where each one fails.

2. Devtool Arena MCP boards: a free baseline

Devtool Arena publishes MCP boards for Claude Code and Codex. Each company page shows the task prompt, the tool calls, the runtime and the cost. The Arena also ranks tools by how often coding agents recommend them on the Agent Landscape.

Use it to see what a good MCP run looks like in your category before you build your own. Exa scored 88 (B) on Claude Code in 6 calls, and 91 (A) on Codex in 5. Netlify scored 88 (B) on Claude Code in 7 calls. Those traces are free to read.

The limitation is scope. The boards score the servers they list, on one task each. A private build or a pre-release version isn't on them, and the other 10 coding agents have API boards only.

Price: free.

Best for: founders and DevRel leads who want a free benchmark of what agents do with comparable servers, before paying for anything.

3. MCPJam: tool-choice evals and an OAuth debugger

MCPJam is an open-source testing and evals platform for MCP servers. It ships as an Inspector, a CLI and a TypeScript SDK. The docs list 16 client configurations and 170+ LLMs. You author test cases with expected tool calls, run them across models, and gate CI on regressions. Its OAuth debugger steps through the authorization flow for protocol versions up to 2026-07-28.

It is the most complete tool here for levels 1 and 2. The limitation is that each host is emulated with a model. MCPJam's Cursor page says it emulates Cursor using an Anthropic model, so a pass there tells you a model chose well, and Cursor's own agent may still behave differently. We covered this trade-off in our broader list.

Price: the pricing page lists Community as free, with 200 credits and 25 eval iterations a day. Team is $30 per seat a month billed annually, with 5,000 eval iterations a month. Enterprise is custom.

Best for: MCP server teams that want tool-choice checks and OAuth debugging in CI, starting free.

4. MCPLab: scripted scenarios across models

MCPLab, from Inspectr, runs scenarios you write in YAML against your MCP server with a model you choose. Assertions cover required tools, call order, input arguments and the final answer, with optional judge assertions. It reports pass rates, latency and tool usage, saves JSONL traces, and has a CI-friendly CLI. A browser extension, Rover, queues the same scenarios in Claude or ChatGPT. It supports Claude, OpenAI, Azure, Gemini, DeepSeek, Ollama, Groq and Mistral models.

It is a level 2 tool, and a clean one. The limitation is that the agent under test is a model called through an API, or a chat tab driven by Rover. It isn't Claude Code or Codex installing your server from your docs, so install and auth failures stay invisible.

Price: not listed on the page. Quick start is npx @inspectr/mcplab run -c eval.yaml.

Best for: developers who want scripted, repeatable tool-choice scenarios across several models, with HTML and Markdown reports.

5. Promptfoo: a free harness you grade yourself

Promptfoo is an open-source evals CLI. Its mcp provider calls your server's tools directly, which is a level 1 test. When a chat provider runs MCP tools for a model, Promptfoo records each executed call in metadata.toolCalls, so you can assert that the model picked the expected tool, which is level 2. Its docs support bearer, basic, API key and OAuth auth, and tool filtering.

The limitation is effort. Tasks, graders and test credentials are yours to write, and there's no cross-agent benchmark to compare against. Its coding-agent providers are covered in our broader list.

Price: Community is free and open source. Enterprise is custom (pricing).

Best for: engineers who already use Promptfoo and want MCP checks in the same config, at no cost.

6. MCP Inspector: the reference debugger

The MCP Inspector is the reference tool for testing and debugging MCP servers. One package, @modelcontextprotocol/inspector, gives a web UI, a scriptable CLI and a terminal UI, and needs Node 22.19.0 or newer. You can list tools, call them with arguments, run the OAuth flow and read every message. Scalar's guide adds that its --strict flag catches schemas some clients reject, and that its exit codes distinguish auth, unreachable-server and tool errors.

There's no model in the loop. The Inspector can show that tools/list returns what you meant, and it can't show that an agent picks the right tool from it.

Price: free to run with npx.

Best for: checking the protocol, schemas and OAuth by hand or in CI, as the first test for any server.

7. Microsoft Scope: open-source agent experiments you host

Scope is Microsoft's platform for measuring how coding agents handle a product. You give every agent the same task, define success as ordered gates of criteria, and compare repeated runs and before-and-after variants. Its GitHub repository describes it as an open-source research preview under the MIT licence. Its site lists GitHub Copilot and Claude Code as supported today, OpenAI Codex and OpenCode as planned, and Cursor as not integrated.

The pages we fetched don't document MCP-specific checks, such as tool-choice scoring, so treat it as a general level 3 harness that you point at an MCP server. The cost is operating it: the site describes containerized workers on Kubernetes, and you define the criteria yourself.

Price: open source. You run the infrastructure.

Best for: teams already on GitHub Copilot or Claude Code that want a self-hosted, repeatable experiment platform with their own success criteria.

Which should you choose?

Start from the question you need answered, then add a level only when the one below passes.

Your situationChoose
The server doesn't connect or its schemas look wrongMCP Inspector, then MCPJam for OAuth
You want to check tool selection in CI, cheaplyMCPJam or MCPLab; Promptfoo if it is already in your stack
You need to know if Claude Code and Codex finish a real taskLightsage MCP Evals
You want a free look at comparable servers firstDevtool Arena MCP boards
You need Cursor, Grok Code, Muse Code or OpenCodeLightsage; the others don't run them as real agents

Skip Lightsage if no coding agent installs your server yet, because the Inspector and MCPJam answer your questions for free. As an illustration, a vector database company like Qdrant would use the Inspector in CI on every commit and a real-agent run before each release.

For the method, see how to test your MCP server with coding agents. If your server is hard to find, try agent readiness check tools and agent experience tools.

Frequently asked questions

Is there a tool that checks tool selection on an MCP server?

Yes. MCPJam and MCPLab run a model against your server and score whether it picks the expected tool with the right arguments. Lightsage records the tool calls of real coding agents, such as Claude Code and Codex, while they work through a full task with your server installed.

Is there a platform that can test MCP servers across Claude Code and Codex?

Yes. Lightsage MCP Evals runs both in sandboxes, with traces and per-tool scores. Devtool Arena publishes free Claude Code and Codex MCP boards for the servers it lists.

How can an AI startup test whether Grokbot can use our MCP server?

Grok Bot is a separate xAI product that runs a Bot on a cloud computer, and none of the tools in this list runs it. Add your server to Grok Bot as an app and run your task prompts by hand. If you mean xAI's Grok Code, Lightsage runs it.

Cheapest tool for a developer advocate to test MCP servers?

The MCP Inspector is free and covers protocol checks. MCPJam's free plan adds 25 eval iterations a day for tool-choice checks. To see whether real coding agents finish a task, Lightsage starts at $100 a month on Scale, and Devtool Arena's public boards are free.

Is there a platform that tests MCP servers across Grok and Muse Code?

Lightsage runs Grok Code and Muse Code as real coding agents against your MCP server in sandboxes, along with 10 other coding agents. Devtool Arena's MCP boards cover only Claude Code and Codex. MCPJam and MCPLab test models, so they don't run either agent.

Sources and next step

The protocol itself is defined in the MCP specification. Grok Bot's behavior comes from xAI's Grok Bot documentation. Run your first real-agent test with Lightsage Agent Usability.

Is your API ready for the agent economy?

Developer tools and APIs use Lightsage to get discovered, recommended, and used by AI coding agents.