Radar MCP vs kubectl: A Kubernetes Agent Benchmark on 54 Faults

Updated August 6, 2026. This post originally reported a 52-scenario run on claude-sonnet-4-6. I have re-run the whole thing on claude-sonnet-5 with 54 paired scenarios and fixed a measurement mistake that was understating the speed result. Every number below is from the new run. One of the original headlines - "76% fewer tool calls" - did not survive, and I explain why. You can now replay any of the 54 scenarios yourself.

I wanted to know one thing: when an AI agent debugs a broken Kubernetes cluster, does it matter what tools we hand it?

Most vendors hand-wave this - "our MCP makes your agent smarter." That is not the bar. If you're giving Claude, Cursor, or an in-house agent access to Kubernetes, the tool surface has to beat raw kubectl on accuracy, cost, and safety. I didn't want to ship a claim I couldn't defend, so I ran the same Kubernetes benchmark with the same model on two tool surfaces. One arm drove raw kubectl through a shell. The other drove Radar's Kubernetes MCP server.

Radar won, but not on the axis I first reported.

TL;DR

On 54 real Kubernetes fault-injection scenarios, an agent driving Radar's Kubernetes MCP server reached a correct diagnosis in a median 41 seconds against 154 seconds for the same agent on raw kubectl - about four times sooner. Accuracy was close given unlimited time (91% vs 87% pass rate), but nobody gives an agent half an hour during an incident: cap it at five minutes and it's 91% against 56%, because Radar's slowest correct diagnosis in the entire benchmark was four minutes and kubectl's was twenty-four.

Worth being precise about what won, because it isn't MCP. MCP is a connector; an MCP server that just proxies kubectl returns the same wall of YAML with an extra hop in front of it, and I'd expect that to be slower than the shell, not faster. What won is what came back through the connector: a model of the cluster Radar keeps current. The kubectl agent spends 73% of the incident blocked, waiting on the cluster to answer. Radar spends 6%.

What changed since the first version of this post

Three things, and one of them was my mistake.

I was timing the wrong thing. The benchmark runs in two stages: the agent submits a diagnosis, then goes on to attempt a fix. I had been measuring the whole session, launch to exit, when the claim was only ever about diagnosis. That second stage ran 60 to 250 seconds in both arms and was squashing the gap. Measuring to the moment the diagnosis is submitted - which is also the artifact that gets graded - takes the result from "about half the time" to about four times sooner.

"76% fewer tool calls" did not replicate. On the stronger model the gap narrows to 43% by mean and 19% by median. A better model simply flails less with raw kubectl. I am retiring that headline: counting tool calls weights a 0.1-second cache read the same as a 60-second rollout wait, which makes it a poor proxy for anything a user feels. Time to a correct answer is the honest metric, and it is the one that got stronger.

Better model, more scenarios. claude-sonnet-5 instead of claude-sonnet-4-6, and 54 paired scenarios instead of 52.

What I tested

The setup was deliberately simple: same benchmark suite, same model, same goal, two tool surfaces.

• Scenarios: SREGym - 54 paired fault-injection scenarios on a live cluster. Bad DNS policies, wrong selectors, missing env vars, broken probes, auth disruptions, stale secrets, admission webhooks, slow image pulls. Real faults, not puzzles.
• Cluster: EKS, 3 nodes, us-east-1.
• Model: claude-sonnet-5 on both arms.
• Tool surfaces: Arm A got raw kubectl via Bash, including exec. Arm B got Radar's MCP tools with kubectl blocked outright, so it could not fall back to a shell.
• Grading: SREGym's own LLM judge at temperature 0. The graded artifact is the first diagnosis each agent submits.

The numbers

The accuracy rows are close enough that I would not lean on them. The timing rows are not close, and they are consistent: across the 44 faults both arms diagnosed correctly, Radar answered sooner in 43.

Accuracy you can actually wait for

Reporting accuracy and speed separately understates what is happening, because nobody lets an agent grind for thirty minutes while production is down. Put the two together - how much of each arm's accuracy is reachable inside a given time budget - and the near-parity disappears:

Radar delivers every correct answer it will ever produce within five minutes; its curve is flat after that. Capping the agent at five minutes costs Radar nothing and costs kubectl more than a third of its correct answers.

Why this Kubernetes MCP server beats raw kubectl - and why not all of them would

This is the part that explains the numbers instead of just reporting them.

Radar's MCP is not a kubectl wrapper - that distinction is the whole mechanism. Give an agent raw kubectl and a single kubectl get pod -o yaml hands back hundreds of lines: managedFields, full status blocks, annotations, the entire object as the API server stores it. The one useful signal - an OOMKilled condition, a failing readiness probe - is buried in the wall of text. And the agent re-reads its accumulated context every turn, so by call 30 it's dragging a huge transcript of YAML it already parsed.

To trace a cross-cutting fault - "this Service can't reach its pods" - a kubectl agent has to reconstruct the graph by hand: list the Service, read its selector, list pods, check labels, read endpoints, check the network policy, pull events, cross-reference timestamps. Worse, most of that is waiting. Rollout status, log tails, sleep-and-retry loops, and in several trials the agent launched a throwaway pod just to run curl from inside the cluster, at roughly two minutes a go. That is where the 73% figure comes from.

Radar hands over the already-computed answer instead. The MCP returns minified, enriched, secret-redacted data: pre-built topology graphs, problem-correlated timelines, deduplicated events, error-filtered logs. It calls the issues API and gets the causal chain - what broke, what it's connected to, what changed - instead of the raw materials to derive it. Those calls return in well under a second, because the expensive work happened continuously, before the incident.

That flips diagnosis from I/O-bound to model-bound. Radar's floor is how fast the model thinks; kubectl's floor is how fast a cluster answers serial commands. Faster models widen this gap rather than closing it.

This is exactly the argument Roy made in "Agents, UIs, and CLIs: The False Choice in Kubernetes Operations" - a cluster is a live graph, and raw YAML is a poor way for anything to make sense of it, human or LLM. This benchmark is that argument with a number attached.

Where Radar pulls ahead, and where it loses

Radar's biggest wins are the faults where nothing crashes. A rotated Secret the pod never picked up, a feature flag flipped in a ConfigMap, a CoreDNS rule changed in another namespace. Every pod reports Ready, describe looks clean, and there is no event to find. The kubectl agent ground for thirty to forty minutes on several of these and still got them wrong. "What changed, and does it correlate with the symptom" is precisely what Radar computes continuously.

We lost five scenarios, and they are all in the replay if you want to watch them go wrong. Two patterns account for most of it.

The first is evidence inside a running process. When a database grant is revoked inside the database and no Kubernetes object changes, the kubectl agent can shell in and read it. Radar has no exec, so it gets the component right and the mechanism shallow.

The second is squarely our bug. The benchmark injects a fault and starts the agent seconds later, while some of Radar's time-based signals are still warming up. The agent asks, gets a short and confident-looking answer, and stops - when the correct answer would have been there two minutes later. Radar needs to say "I have not watched this cluster long enough yet" instead of looking finished. That is on our roadmap because this benchmark found it.

Worth being blunt about the other side too: the kubectl arm's distinctive wins rest on giving an autonomous agent free-form exec on your cluster. Almost no security team grants that, and they are right not to.

You don't want an agent loose on raw kubectl

Speed and accuracy aside, raw kubectl through a shell is the ungoverned option. The agent inherits whatever that shell can do, every call is a potential mutation, and you get no record of what it touched. This benchmark contains a fault whose most natural diagnostic command prints a production password in plaintext, straight into an LLM context window.

Radar's MCP is the opposite by design: read tools are read-only and secret-redacted, writes are RBAC-enforced and gated so the client can prompt before anything changes, and Radar Cloud adds the governance layer on top - an inventory of every agent and token with cluster access, the same Kubernetes impersonation a human gets, and audited tool activity for agent actions.

"Faster and a little more accurate" is the benchmark result. "And you can actually let it near production" is why teams route the agent through Radar in the first place.

Why this is the argument for an agent surface

Step back and the result isn't really about AI - it's about data shape. Agents reason over Kubernetes far more cheaply, and a little more accurately, when the data is structured for them. It's the same reason we build dashboards instead of reading raw YAML: eyes need the graph rendered, and it turns out agents need the same graph modeled.

So Radar is one engine that watches your clusters, with two surfaces over it. Humans consume it as a UI - still the surface most of our users live in. Agents consume it as MCP. Both need the same thing: the cluster made sense of, not just dumped. This benchmark was about the second surface, but the human-first one is still the point - we build Radar for the person staring at a cluster at 2am, and increasingly some of that work is done by an agent on their behalf.

Watch it, then run it yourself

You don't have to take my word for any of this. Every one of the 54 scenarios is replayable side by side, call by call, including the ones Radar got wrong:

Replay the benchmark →

And Radar is open source with the MCP server on by default, so you can point your own agent at it and break something on purpose to see how it reasons.

curl -fsSL https://get.radarhq.io | sh && kubectl radar

That gets you the UI and the MCP server in one go. Aim your agent at the MCP endpoint and watch it work a fault on a cluster you trust.