Radar Is #1 on SREGym: 95% of Live Kubernetes Faults Fixed

As of November 5, 2026, Radar + Claude Code is first on both SREGym leaderboards. SREGym is an open, academic benchmark for AI SRE agents: it deploys a real application on a live Kubernetes cluster, injects a fault, and grades whether the agent finds the root cause and fixes it in the same run.

On the full suite (SREGym-0508, 90 faults, three attempts each):

That is 23 points ahead end to end, about 6x faster to a diagnosis and to a fix, on roughly a quarter of the tokens. The result is verified by the SREGym team. We haven't run the separate "with noise" board yet.

Why SREGym

SREGym's problems look like real incidents. The 90 on the full board go well past misconfigured YAML: OS-level faults like a crashed kubelet or silent data corruption, a retry storm, and several failures at once. Some SREGym problems recreate public postmortems: the Lite board includes a simulation of Cloudflare's 2019 WAF regex outage, and the wider problem set covers others, such as Reddit's Pi-Day outage. It builds on two earlier research benchmarks, AIOpsLab and ITBench, and it grades the fix against the live system, not only the agent's write-up.

It's also becoming the shared yardstick for AI SRE. The paper is at NeurIPS 2026, GitHub Copilot and Devin are on the board, and Datadog used SREGym as its outside check when it introduced Scout 1.0, the model behind Bits AI, in October.

And it's run carefully. It's MIT-licensed, so anyone can rerun a result. The SREGym team verifies third-party submissions before listing them, and retires problems once agents saturate them, so the board keeps getting harder. They also reviewed and merged our client upstream, which is why you can rerun this one. We'd like to see more AI SRE tools on the public board, so results can be compared on the same faults with the same grading.

Radar was the agent's only way into the cluster

The agent container had no kubectl and no kubeconfig. Every read and every fix in all 270 runs was a Radar MCP call. We set it up that way on purpose: otherwise you can't tell whether a result comes from Radar or from the shell.

• Agent: Claude Code on Claude Opus 5.5 (medium effort), with SREGym's task prompt pointed at Radar's tools instead of kubectl.
• Radar: v1.19.0, started fresh for every run.
• Grading: SREGym's own, unmodified. No run was retried or dropped.

The client is upstream in SREGym as clients/claudecode_radar (#1144), so anyone can rerun it (see the end of this post).

Example: broken cluster DNS, fixed in 38 seconds

In one fault, SREGym breaks the cluster's DNS. It edits the config of CoreDNS, the cluster's internal DNS server, so that every internal service name stops resolving. In the demo app, an online shop, every service starts failing to reach every other service.

What makes this one hard is where the cause lives. The agent is pointed at the shop's namespace, and that's where all the errors are: failed requests and connection errors in service after service. The cause is a config change in kube-system, a namespace the task never mentions. An agent that starts with the shop's pods and logs finds plenty of errors, and every one of them is a symptom.

The agent's first move was to ask Radar what was wrong in the shop's namespace. Along with the shop's errors, Radar's answer included this:

ConfigMap kube-system/coredns is suspicious (CoreDNS NXDOMAIN override); Cluster DNS
context may be relevant: inspect kube-system/CoreDNS before attributing DNS or
service-resolution symptoms only to application workloads.

In plain terms: the cluster's DNS config was changed to answer "no such name" for everything, so check that before blaming the app. Radar connected the shop's symptoms to a change outside the shop before the agent had read a single log line.

From there, every step was a Radar call:

1. Checked what had changed recently and read the CoreDNS config. Diagnosis submitted at 21 seconds.
2. Patched the config to remove the bad block.
3. Restarted CoreDNS and waited for it to come back.
4. Checked from inside one of the shop's pods that service names resolved again. Fix submitted at 38 seconds.

Its diagnosis named the exact block that broke DNS and said the shop's services were victims, not the cause. All three attempts at this fault passed.

The tools the agent used across 270 runs

The agent used 20 of Radar's 37 tools. Every run started with issues. Most fixes were a targeted patch_resource, not a re-apply of a whole manifest, and most runs checked their own fix with wait_for or exec_pod before submitting. Agent cost was $0.41 per run.

Same model, other setups

The entries on the full board use different models, and no other entry runs Opus 5.5, so the board alone can't separate Radar's contribution from the model's.

Datadog's Scout 1.0 report helps here. It ran newer models on the same 90 faults with SREGym's own agent harness, one attempt each, outside the public board. The best was GPT-6.1 Sol at 82 of 90, and Claude Opus 5.5 at xhigh effort fixed 79. With Radar, Opus 5.5 at medium effort fixed 88, 85 and 84 in our three attempts.

On SREGym-Lite, Devin Oncall ran the same model at the same effort:

Five Lite entries tie at 100% end to end. Among those five, the differences are time and tokens.

What it missed

It missed 13 of 270 runs, and in most of those the diagnosis was right but the fix wasn't. The pattern is clear: faults where a Kubernetes object holds the cause, like a wrong Service selector or an RBAC misconfiguration, closed in under a minute. Faults that need evidence from inside the application or the node, like Kafka queue problems or a crashed kubelet, took 3 to 10 minutes, and that's where all 13 misses were.

Radar-only was a choice, and you can make it too. An agent with a raw kubeconfig can run any command its credentials allow. With Radar as its only access, every read and change is a named tool call: Kubernetes RBAC still applies, structured reads leave out Secret values, deletes need a preview and a confirmation token, and your MCP client can ask before any write. For an agent near production, that is often the better default anyway.

We tested it both ways, with kubectl allowed alongside Radar and with only Radar. The board entry is the Radar-only run.

Run it yourself

The full client setup is in SREGym's clients/claudecode_radar README: a filtering proxy so Radar sees exactly what a kubectl agent would, and a fresh Radar per problem. Then:

uv run main.py --suite sregym-lite --agent claudecode_radar \
  --model anthropic/claude-opus-5-5 --reasoning-effort medium \
  --allow-agent-endpoint http://host.docker.internal:9283 --force-build

To give your own agent the same context against your own cluster:

brew install skyhook-io/tap/radar
kubectl radar
claude mcp add radar --transport http http://localhost:9280/mcp

exec_pod is off by default; start Radar with --mcp-exec-tools to enable it.

Radar is open source (Apache-2.0), and every tool used in these runs ships in the open-source binary. The MCP page lists all 37 tools, and our benchmark hub has the earlier diagnosis-only studies, with per-fault data.