> ## Documentation Index
> Fetch the complete documentation index at: https://radarhq.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Reachability
> Did traffic actually get through? Trace the declared network path, test it live from multiple vantages, and see exactly which hop breaks - without ever claiming more than was verified.
The **Reachability** tab in the resource detail view answers one question for a network entry kind:
> *If traffic is sent toward this resource, does it reach a healthy process - and if not, which hop is the first to break?*
Available in Radar **v1.9.1+** (the in-cluster probe test also requires the probe image from v1.9.1+).
Naming note: this is NOT the [AI Diagnose feature](/docs/features/mcp) - Reachability is deterministic path tracing and live probing, no AI involved. The general-purpose MCP `diagnose` tool (whose primary job is workload root-cause bundles: logs, events, crash evidence) returns this reachability trace as its answer when pointed at a network entry kind - see [MCP](#mcp).
The trace has two layers:
1. **Static** - is the path wired correctly in config + current pod state? Pure functions over the in-memory informer cache, no per-call API requests. Always on.
2. **Active reachability test** (optional, one-shot) - send DNS / TCP / TLS / HTTP probes along the declared path and report what came back. HTTP-shaped ports get an HTTP request; explicitly non-HTTP ports stop at TCP rather than sending an unrelated protocol. The proxy probe runs automatically once when the **Diagnose** tab opens (re-runnable via **Run test**); only the in-cluster Job test stays a manual click.
The active layer can escalate the static verdict when probes give clear evidence of a real failure on a hop (every non-skipped probe failed → that hop counts toward broken; over half failed → counts toward degraded). It never softens a static verdict: a critical static finding outranks probe state, and an unverifiable path stays unverifiable.
Probes run from wherever Radar is running - your laptop, or inside the cluster - and where they run from changes what they can prove. When a probe fails for a reason that belongs to the vantage rather than the workload, Radar marks the hop and leaves the verdict at *unknown* instead of reporting the path as broken; only a result from the real traffic path sets the headline. The **Limits** section below lists the cases where this happens and what to do about each one.
*One Service, two declared front doors. Each entry is judged on its own - the HTTPRoute here is not attached to any listener - and the verdict names the vantage it speaks from: reached via the API-server relay, which is not the live-traffic path.*
## Mental model
```
┌─────────────────────────┐
│ Upstreams │ ← parallel hops INTO the subject
│ Ingress · HTTPRoute │ (each judged independently)
└────────────┬────────────┘
│
┌────────────▼────────────┐
│ Subject │ ← the resource you opened
└────────────┬────────────┘
│
┌────────────▼────────────┐
│ Downstream │ ← the chain FROM the subject
│ Service → Pods │ (the broken hop indexes here)
└─────────────────────────┘
```
**Upstreams are parallel.** A Service that's reached by both `ingress-a` and `ingress-b` does not become broken when only one of them fails - the other still delivers traffic. The verdict only degrades to *broken* if every upstream is broken, *degraded* if some are.
**Downstream is a chain.** The first critical finding along Downstream is the broken hop, named `brokenAt`. Findings on later hops are still shown but the diagnosis starts at the first failure.
## Supported entry kinds
| Kind | Downstream chain | Upstreams |
| ------------------------- | ------------------------------------------------------ | -------------------------------------------------------------- |
| **Service** | Service → selected pods | every Ingress / HTTPRoute / GRPCRoute referencing this Service |
| **Ingress** | Ingress → backend Services → pods of the first backend | none (external entry) |
| **HTTPRoute / GRPCRoute** | Route → backend Services → pods of the first backend | every parent `Gateway` from `parentRefs` |
| **Gateway** | Gateway → attached routes (capped at 20) | none |
Other network surfaces in the topology (Traefik IngressRoute, Istio VirtualService, Knative) are deliberately out of scope today - they would each need their own resolution logic and aren't reached by the same `parentRefs` / `backendRefs` shape.
## What each hop carries
Every hop has:
* A **resource reference** (kind, namespace, name) and an **edge label** (e.g. `HTTPRoute->Service`) so the path reads top-to-bottom in the traffic direction.
* **Findings** - the detections that already exist in the issues pipeline, attached to the hop where the failure is observable. The Phase 0 additions to this pipeline:
* **`gwroute:backend-port-mismatch`** - an HTTPRoute / GRPCRoute references a Service port that doesn't exist.
* **Gateway-API route parent conditions** (`Accepted=False`, `ResolvedRefs=False`, `Programmed=False`) - read directly from `status.parents[]` so the controller's own verdict is the source of truth.
* **Meta** - pod counts (`selected` / `ready`), `endpointSource: pod-readiness`, `headless`, and `selectorless` flags so the UI can render the right shape without re-deriving them.
For each finding the trace populates a **kubectl reproducer command** - a one-liner the operator can paste to see the raw state behind the finding. Examples:
```bash theme={null}
kubectl describe service api -n prod
kubectl get pods -n prod -l app=api
kubectl get httproute api-route -n prod -o jsonpath='{.status.parents}'
```
## Reachability test (active probes)
The proxy-vantage test runs automatically when the tab opens and is re-runnable via **Re-run reachability test**; the **Test in-cluster** button (consent-gated - the dialog names the exact requests the probe pods will send, e.g. "TCP connection to redis-master:6379") runs the probe Job. Each run fires one round of applicable probes. HTTP(S) routes use their declared path; non-HTTP Service ports have no HTTP path and stop at TCP:
| Hop | What runs |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Ingress / Gateway hostname | DNS → TCP → TLS (if HTTPS) → HTTP |
| Service | Direct TCP to ClusterIP:port in-cluster, followed by HTTP only for an HTTP-shaped port; from a laptop, HTTP via the K8s API server's `/services/{name}:{port}/proxy/` subresource when applicable |
| Pods | Direct TCP to PodIP:port in-cluster, followed by HTTP only for an HTTP-shaped port; from a laptop, HTTP via `/pods/{name}/proxy/` when applicable for up to 3 sampled ready pods; the remaining pods get a "sampled N of M" skip row |
| HTTPRoute / GRPCRoute | Skipped - routes have no own routable address; reachability is the upstream Gateway + downstream Service |
Each row reports outcome (`ok` / `fail` / `skipped`), latency, the path it traversed (`pod-to-pod path` or `via Kubernetes API`), and an HTTP status detail when available. The total budget is 3 seconds; per-hop runs in parallel within that envelope. Probes are an action, not a polling state; the button fires once, results land, the next static refetch replaces them.
**Vantage is a first-class concept in the UI.** A reachability result is meaningless without knowing where the request came from, so the tab is built around two explicit selections: WHICH path (the **PATH** picker on the graph - one row per declared route, with per-port protocol labels like `port 53/UDP (dns)` when a Service declares dual-protocol or non-HTTP ports) and FROM WHICH vantage (selectable capsules on the graph: *Radar on your machine* dialling as a client, the *API-server proxy* relay, and the *in-cluster probe*). Selecting a vantage genuinely re-routes the graph and re-scopes the verdict, the edges, and the inspector - a laptop's success is never painted onto the in-cluster lane. Vantages Radar cannot use (a real caller workload, a genuine external client) stay visible as stated gaps rather than being hidden, so synthetic evidence never looks complete.
Every claim on the page carries its evidence: the verdict band states the live-check volume ("8 live checks from 2 vantages", with the DNS/TCP/TLS/HTTP breakdown on hover), a skipped route states WHY it was skipped from the exact vantage that skipped it, and node dots show each resource's own health - cluster state, never a test result (edges carry the test truth).
*A non-HTTP Service (Redis on 6379) after an in-cluster test: the throwaway Pod connected over TCP, and the coverage line states the ceiling of that proof - "TCP connections only - application protocol not checked".*
*kube-dns declares TCP and UDP on the same number. Each is its own declared path: the TCP route is testable, and the UDP one stays a stated gap rather than being folded away.*
*A front-door subject. Radar dials the declared hostname from your machine (DNS, TCP, TLS, then HTTP) - here a 308 redirect - while the headline stays honest that a backend-only confirmation does not exercise the entry path.*
## Security model
The active layer is the only part of Reachability that puts traffic on the wire. What it can reach is bounded by Kubernetes, not by Radar.
**Probes run as the requesting user.** Radar has no permission system of its own - it delegates to Kubernetes RBAC by impersonating the signed-in user (see [Authentication](/docs/configuration/authentication)). A probe can only reach what that person could already reach with kubectl, and the API server evaluates the request under their identity. If impersonation can't be established, the probe fails rather than falling back to Radar's own ServiceAccount, which would silently widen the caller's reach. (In the local single-user binary, or a deployment running `auth.mode=none`, there is no user to impersonate and Radar acts as its own credential; per-user scoping begins when authentication is enabled.)
**Two separate permission gates**, depending on how the test runs:
| How the test runs | RBAC checked in the target namespace |
| ---------------------------------------------- | ---------------------------------------------- |
| Relayed through the API server (from a laptop) | `get services/proxy` and/or `get pods/proxy` |
| From inside the cluster, as a probe Job | `create jobs`, `list pods`, and `get pods/log` |
Radar running in-cluster dials the data path directly and needs neither proxy verb. To switch the active layer off, remove those verbs from the identity Radar impersonates - there is no application-level flag to bypass, because the API server is what enforces it. A denial is reported as a denial ("your identity lacks `get services/proxy` in this namespace"), never scored as an unreachable service.
**The in-cluster test creates something, so it asks first.** Nothing is created until someone clicks, and the consent dialog names the exact requests the probe will send (for example, `TCP connection to redis-master:6379`). The three verbs above are checked with a `SelfSubjectAccessReview` *before* anything is created; if any is missing, Radar returns a copyable `kubectl run` mirroring what it would have done rather than half-running. On Radar Cloud the probe additionally requires org role Member or higher.
The probe pod itself is built to be the least interesting workload in the namespace:
* At most 5 per run, one per intended route
* **No ServiceAccount token mounted** - the probe holds no Kubernetes API credentials at all
* Non-root (UID 65532), read-only root filesystem, all capabilities dropped, no privilege escalation - it satisfies the `restricted` Pod Security Standard
* 50m CPU / 32Mi memory, request equal to limit; no retries
* Killed at 25 seconds, deleted right after the run, TTL-reaped 60 seconds later as a backstop
### What an agent gets
Reachability reaches agents through the same MCP `diagnose` tool described below, and the shape of that surface is the point.
An agent talking to Radar over MCP does not hold a kubeconfig and cannot run kubectl. It calls typed tools with fixed schemas, fixed at build time, with no raw-request, `exec`, or shell escape hatch to fall back on - and its calls go out under the same impersonated identity as the human driving it, so RBAC bounds the agent exactly as it bounds them and the audit log names a person rather than a shared robot account. Radar also serves a strictly read-only MCP endpoint at `/mcp-readonly`, where the mutating tools are never registered: they aren't gated behind a check an agent could be talked past, they don't exist on that endpoint.
Within that surface, Reachability is one of the narrowest capabilities you can delegate. It reads a declared path and sends probes along it. The only object it can create is the probe Job above - capped, credential-less, self-deleting - and only when `inCluster: true` is passed explicitly. Compare that with the usual alternative of handing an agent a kubeconfig and a shell, where the capability surface is every verb on every resource and the only limit is the model's own judgement.
## What it deliberately does NOT do
* **No EndpointSlice reads.** The endpoint signal is pod-readiness; the trace marks this with `endpointSource: pod-readiness` so the approximation is honest.
* **NetworkPolicy is predicted, never enforced by us.** Radar statically evaluates the *caller-independent* ingress rules of core NetworkPolicies that select the subject's pods. When no rule admits the path's port it surfaces a "a cluster network rule would block traffic to these pods" **WARNING prediction**; when a rule allows the port only from specific sources it surfaces a source-restricted **advisory** (Radar can't tell from its own vantage whether *your* caller is allowed); an egress-only policy becomes an outbound **note**. This is always a PREDICTION, not a verdict - the CNI is the only authority on enforcement (some plugins write the NetworkPolicy object but enforce nothing), so the live in-cluster probe confirms the would-block or downgrades it when real traffic got through. We do not model CNI-specific enforcement behavior.
* **No external-path probing** for Service type LoadBalancer / NodePort - the external path requires modelling cloud LB state honestly and is intentionally out of scope today. (ExternalName **is** probed: Radar DNS-resolves and HTTP-reaches the alias host from its own vantage, marked *indirect* - a reach from Radar's network is not proof of in-cluster reachability.)
* **No new CRDs.** Everything reads from the same informer cache the rest of Radar uses.
These are not gaps to fix soon - they are the line that keeps the feature trustworthy in clusters Radar cannot fully see.
## Limits - when an active probe can fail on healthy traffic
A probe reports what *Radar's vantage* observed. Real traffic can flow fine while a probe fails, for reasons that have nothing to do with the workload:
* **Service mesh mTLS (Istio / Linkerd / Consul).** A mesh with strict mTLS rejects any connection without a valid mesh client certificate. A probe from your laptop (or any caller outside the mesh) has no such cert, so the TLS/HTTP layer fails even though sidecar-to-sidecar traffic is healthy. Recognize it: the pods carry a mesh sidecar (`istio-proxy`, `linkerd-proxy`) or mesh labels (`istio.io/rev`, `linkerd.io/inject`). Trust real in-mesh traffic over an out-of-mesh probe.
* **NetworkPolicy.** A policy can allow real workload-to-workload traffic while blocking Radar's API-server-proxy identity (or the reverse). A failed probe next to healthy config is often this. The trace statically predicts a would-block from the caller-independent ingress rules, but the CNI is the only enforcement authority - so the prediction is confirmed or downgraded only by the live in-cluster probe.
* **DNS split-horizon.** A probe resolves hostnames from *your* vantage, not the cluster's. An internal name that resolves inside the cluster may not resolve (or may resolve differently) from your laptop; those probes skip with a reason rather than failing.
When config is healthy but a probe fails, suspect the vantage before the workload. Run the **in-cluster** test (or `diagnose(inCluster: true)`) to probe from the real dataplane, where mesh certs and in-cluster DNS apply.
## Verdict semantics
The verdict is a **coverage claim over the intended routes** - what was actually tested, not what the config implies. It judges ONLY the intended traffic route: an API-server-proxy probe can *localize* a failure but never *sets* the headline - only a real-traffic (data-path) result does. Each tested route carries an **outcome** and a **confidence**:
| Outcome | Meaning |
| ---------------- | -------------------------------------------------------------------------------------------------------- |
| **verified** | real HTTP 2xx - the route works |
| **reached** | reached (3xx/4xx, or transport-only TCP/TLS) - the server answered, but the route itself isn't confirmed |
| **server-error** | reached, but the backend returned 5xx |
| **unreachable** | transport failure - nothing answered |
| **not-tested** | no probe could run for this route (it was skipped) |
| Confidence | Meaning |
| ------------ | -------------------------------------------------------------------------------------------- |
| **real** | tested the way real traffic flows (direct dial / in-cluster data path) - can set the verdict |
| **indirect** | reached only via the API-server proxy - annotates a route, **never** a confident green |
The route outcomes roll up to the verdict:
| Verdict | When |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **healthy** | every tested intended route was reachable over **real** traffic |
| **degraded** | reached-but-erroring (5xx), or some routes failed, or an intentional scale-to-0 (benign - reads amber, never red) |
| **broken** | a genuine `unreachable` on the intended route |
| **unknown** | nothing could be actively tested, OR the route was reachable **only** via the API-server proxy (the real path was never confirmed) - never a confident green. Also when the subject isn't in the cache, RBAC denies the namespace, the relevant API isn't installed, or the cache hasn't synced. |
A real-traffic `reached` non-2xx (e.g. a 404) reads as healthy **coverage** with a qualified headline - *"server reached, route not verified"*: the network path is reachable, but the route itself isn't confirmed.
`verdict` is therefore a **coarse rollup**, not a promise of a verified 2xx. `healthy` means "no failing route was found" - it can include routes that were only *reached* (a 3xx/4xx, or a transport-only TCP/TLS connection) rather than verified, and can reflect a static-config-only assessment (`probe=false`) or partial coverage. Agents (and the MCP `diagnose` tool's `verdict` field) that need certainty must read each route's `outcome` (verified vs reached vs not-tested) and `confidence` (real vs indirect) plus the `headline` / `diagnosis` text before keying an action on `verdict` alone.
The UI shows the verdict at the top of the panel with a one-sentence reason. Treat *unknown* as a pause-and-investigate signal - it means the trace can't honestly answer the question, not that everything is fine.
## MCP
The general-purpose `diagnose` MCP tool - primarily a workload root-cause tool (logs, Warning events, crash evidence in one call) - returns the reachability trace for network entry kinds instead of the pod-log fan-out it does for workloads. An agent that calls `diagnose(kind=service, ...)` gets the path-shaped answer in one call, along with `relatedIssues` for raw-issue follow-up. Pass `probe: true` to add the active reachability test from Radar's vantage. Pass `inCluster: true` to run the probe from inside the cluster - Radar creates up to 5 short-lived, self-destructing probe pods (one per intended route) under the caller's RBAC to test the real dataplane the API-server-proxy vantage can't reach (e.g. to confirm a route that came back `indirect`). This is the only mutating `diagnose` option; it needs `create jobs` + `list`/`get` pods RBAC, and falls back to a copyable command when pod-create is denied.
## In-cluster probe image
The in-cluster test runs a short-lived Job whose container executes `radar probe`. Radar resolves which image that Job uses, most-trusted first:
1. **Explicit override** - `--reachability-image` flag / `reachabilityImage` config. Always wins.
2. **Self-read** - radar's OWN running pod image, read from `MY_POD_NAME` / `MY_POD_NAMESPACE` (downward API) using radar's service account. The honest default: the probe runs the *same* image as radar, so it's automatically correct for private registries, mirrors, and digest-pinned deploys.
3. **`RADAR_IMAGE` env** - the Helm chart sets this to radar's deployed image; a no-RBAC fallback for when self-read can't run.
4. **Version-matched published image** - `ghcr.io/skyhook-io/radar:`, where `` is radar's own build version.
The image must ship the radar binary at `/radar` (the Job runs `["/radar","probe",...]`) - every official radar image does.
By deploy shape:
* **Standalone (Helm chart):** self-read or the chart's `RADAR_IMAGE` resolves radar's deployed image. Nothing to configure.
* **Cloud agent (`--cloud-url`):** radar runs in-cluster as its own image; self-read resolves it. Nothing to configure.
* **Embedded in another app:** embedding only the *frontend* library (e.g. Radar Hub) is unaffected - the backend is still a real radar image. If radar's *backend* is wrapped inside a non-radar image, self-read returns that host image, which only works if it ships `/radar` with the `probe` subcommand; otherwise set `--reachability-image` / `RADAR_IMAGE` to a published radar image. If neither is valid the probe pod fails honestly (a clear "couldn't start" status + a copyable command), never a wrong result.
**Local development:** a `-dirty` local build's version isn't a published tag, so the default (#4) won't exist. Build a probe image from your code and load it into kind:
```
make kind-load-probe # builds radar-probe:dev and kind-loads it
radar --reachability-image radar-probe:dev ...
```
## Performance
**Static trace:**
* No per-call API requests; pure functions over the in-memory informer cache
* Linear in path length + selector match counts, which are bounded by the cache contents
* Target \<100ms typical, \<300ms on a 200-pod namespace
* Five-second polling from the UI while the Reachability tab is open; data is cached client-side so retabbing within the drawer is instant
**Active reachability test:**
* Budgeted at 3 seconds total; per-hop probes run in parallel within that envelope
* Each layer respects a strict per-call timeout (DNS 250ms, TCP 700ms, TLS 1s, HTTP 1s) so a single dead hop can't starve the rest
* Triggered by an explicit operator click; no polling, no sticky on-state
## Further reading
For the story behind the design - why a probe can fail on a perfectly healthy path, and how the verdict model keeps synthetic evidence honest - see the [Reachability deep dive](https://radarhq.io/blog/kubernetes-network-reachability) on the blog.