Introduction
Since the Model Context Protocol shipped in late 2024, adoption has been steep. The MCP SDK has reached 97M monthly downloads, public directories and trackers catalogue more than 10,000 active servers, and roughly 38% of SaaS connectors now have an MCP option. Whether MCP ultimately beats a well-documented API is still argued over, either way, it's here to stay, and it's now in production paths.
That's the problem. MCP is new enough that the tooling around it hasn't caught up, and the failure modes are a mix of the novel and the very familiar:
- Authentication gaps. Only about 8.5%1 of deployed MCP servers implement the OAuth 2.1 authentication protocol the specification requires.//
- Leaked configuration. An estimated 53%2 of MCP servers expose sensitive information through hard-coded values in configuration files.
- Stale contracts. Tool descriptions drift from tool behaviour. Agents plan against the description, so a wrong description is a wrong plan.
- Unenforced boundaries. A tool that accepts a path, a workspace, or an ID often doesn't check that the caller was entitled to it.
None of this is visible from reading a schema. An MCP tool's real contract is what it returns when you call it, so that's where Kerno tests it.
How it works
Kerno introspects a running server, plans a set of scenarios per tool, implements them in a local sandbox, and reports what actually happened. You (or your agent) approve the plan before anything runs.
.gif)
Worked example
We pointed Kerno at our own server and asked for a black-box test of kerno_healthcheck. Kerno brought the app up so runtime behaviour could be exercised, then proposed five scenarios: the happy path, an explicit workspace path, an invalid path, a wrong-typed path, and an unknown extra argument.
From here, Kerno executed the scenarios from the Kerno validation sandbox against the live request. Four passed. One was blocked. And one pass came with a flag attached.
Potential bugs found:
Evidence
The invalid workspace path (/tmp) returned a normal result instead of an error, meaning the tool wasn't enforcing its own workspace boundary. Kerno confirmed it two ways rather than taking the live result at face value: the same argument was correctly rejected by a sibling tool, and the source showed the unvalidated helper still in use at a small number of call sites. Severity was limited, since the surface is read-only diagnostics, but an arbitrary local path was reachable, and the response disclosed whether a .git directory existed there.
Two smaller findings came out of the same run. The tool's description claimed a 120-second wait that doesn't exist in the code, so any agent planning timeouts from that description is planning wrong (calls return in under a second). And a mismatch error from a neighbouring tool leaked the workspace path in plaintext.
Three real defects, from five scenarios, on a tool we wrote ourselves and believed was fine.
Why it matters
Hand-rolling a harness for every MCP tool you ship is work nobody budgets for, so mostly it doesn't happen, and the tools go out untested. Kerno replaces that with runtime evidence: scenarios generated from the tool's actual surface, executed against a live server, findings verified against source before they reach you. You find the boundary bug, the stale description, and the leaked path before a customer's agent does.
MCP is going to carry more and more production traffic. It should be tested like it.
Try MCP testing in Kerno https://www.kerno.io/
Sources:
OX Security Research (2026) The Mother of All AI Supply Chains: Anthropic's "By Design" Failure at the Heart of the AI Ecosystem [Report]. OX Security.
