Skip to main content
Hosted Diagnose is available on a self-hosted control plane. It is off by default. When you turn it on, each investigation turn runs as a short-lived Kubernetes Job in a sandbox namespace, and the model is called through your own provider account. In the chart it is configured under hub.agent.

How it runs

  • Each turn is a Job (pods labelled app=diagnose-turn) in the sandbox namespace. The pod has no mounted ServiceAccount token.
  • The agent reads your cluster only through the control plane’s MCP tunnel, the same path the web app uses. Write tools are refused.
  • The model credential is injected only into a sidecar broker, never into the agent container. Only the selected provider’s broker is mounted, so the pod holds no credential for any other provider.
  • A NetworkPolicy allows DNS, the hub, Postgres, and outbound HTTPS on port 443, and denies all ingress. The HTTPS rule is port-based: it does not limit which destinations the pod reaches. For destination-level control, route the sandbox through an egress proxy or use a CNI policy that filters by FQDN.

Requirements

  • Chart 1.7.0 or later. Earlier charts don’t expose hub.agent. The sandbox image ghcr.io/skyhook-dev/radar-hub-ai-agent-sandbox is public and always uses the chart’s appVersion tag, the same as the hub.
  • A model provider account - one of:
    • Anthropic - an API key (sk-ant-...). Works from any cloud.
    • Amazon Bedrock - an IAM service-specific credential for bedrock.amazonaws.com, with model access enabled in the region you choose.
    • Google Vertex AI - a service-account key (JSON) with roles/aiplatform.user on the project, and the Claude model enabled in Model Garden. Only key-based auth is supported; workload identity is not.
  • A CNI that enforces NetworkPolicy, if you rely on the sandbox egress policy. Enforcement is CNI-dependent: kindnet ignores NetworkPolicy, and on EKS network policy support in the Amazon VPC CNI is off until you enable it.
  • Egress from the sandbox namespace to your provider over HTTPS (api.anthropic.com, bedrock-runtime.<region>.amazonaws.com, or for Vertex oauth2.googleapis.com plus the endpoint for hub.agent.vertex.location: aiplatform.googleapis.com for global, aiplatform.us.rep.googleapis.com for us, aiplatform.eu.rep.googleapis.com for eu, and <region>-aiplatform.googleapis.com for a region such as us-east5) and a way to pull the sandbox image.

What the chart creates

With hub.agent.enabled=true, the chart renders these in the sandbox namespace: <fullname> is the release name when it already contains radar-hub, otherwise <release>-radar-hub. For the release radar-hub used throughout these docs, that is namespace radar-hub-sandbox and Secret radar-hub-agent. Helm won’t install over a namespace it doesn’t own, so a name collision fails the install instead of taking the namespace over.

Provider values

hub.agent.engineLabel sets the engine name shown on the investigation panel (for example Claude (via Agent SDK)). Empty hides it.

AI analysis in alerts

Alert rules can attach an AI analysis to the issues they match. hub.agent.alertAnalysis: true is the install-wide switch for it, and it needs the engine: setting it with hub.agent.enabled: false fails the render rather than sitting in your values doing nothing. Three more gates sit outside the chart, all off until someone turns them on:
  • An organization owner consents. The disclosure appears the first time AI analysis is turned on for an alert rule, and Settings → Organization → Agent access shows the state afterwards.
  • An alert rule opts in - see Alerts.
  • The hub process runs the alerts worker. It does by default; a replica running with it disabled analyses nothing.
Analyses call the same model account as the investigations you start by hand, so the provider spend follows how often your rules fire.

The sandbox DSN

The sandbox pod runs in another namespace, so it can’t use the hub’s own DSN when that DSN is a bare Service name. The chart needs one of:
  • Bundled eval Postgres - set postgres.bundled.auth.password explicitly, and the chart derives the cross-namespace DSN. A generated password can’t be read back when the chart renders, so without an explicit password the install fails (see Add the agent to an existing install).
  • External Postgres - set hub.agent.credentials.podDSN to a DSN whose host resolves and is reachable from the sandbox namespace.
  • Your own Secret - see below.

Using your own Secret

For sealed-secrets or external-secrets workflows, create a Secret in the sandbox namespace and set hub.agent.credentials.existingSecret to its name. Don’t set apiKey, serviceAccountJSON, or podDSN alongside it; the chart refuses the combination. The Secret must carry: A Secret can’t be referenced across namespaces, so a copy in the hub’s namespace doesn’t work.

Enable it on a new install

Take your install command - the one from Install, or the one generated in the web app under Settings → Self-Hosting - and add the agent values. With the bundled eval Postgres, also pin its password:
Keep the credential in a file and pass it with --set-file, so it stays out of your shell history. Write the file without a trailing newline (printf '%s' "$ANTHROPIC_API_KEY" > anthropic.key) - --set-file copies the file byte for byte. For Bedrock or Vertex, swap the provider lines as shown in the next section. With external Postgres, drop the postgres.bundled.auth.password line and set hub.agent.credentials.podDSN instead.

Add the agent to an existing install

The command generated under Settings → Self-Hosting doesn’t set a Postgres password, so on first install the chart generated one and stored it in the <fullname>-postgres Secret. Upgrading with only the agent values fails at render time with:
Pass the existing password back in. Don’t pick a new one: changing postgres.bundled.auth.password doesn’t rotate the password inside the database, so the hub and the sandbox would both lose access.
1

Save the current values

The upgrade below starts from this file, so everything you set at install time (license, break-glass admin, public URL, exposure) carries over:
This file holds every value you passed inline, including the cookie password, license, break-glass credentials, and model key. Keep it readable only by you, never commit it, and delete it (or move it into your secret store) once the upgrade succeeds.
Check helm history radar-hub -n radar-hub first. If the latest revision is failed, the release’s current values are those of the last successful revision, not the failed one. Add --revision <number> to read the values of the revision you actually intended (see Troubleshooting).
2

Read the generated Postgres password

Skip this step if you use external Postgres; set hub.agent.credentials.podDSN in the next step instead.
3

Upgrade with the agent enabled

-f radar-hub-values.yaml carries your install values while the new chart’s defaults, which the agent settings depend on, still apply. Plain --reuse-values would drop those defaults when you move to a newer chart.
If the web Service is a LoadBalancer that hasn’t received an address, leave out --wait; see Troubleshooting.
4

Verify

Follow Verify below.

Verify

  1. The hub logs the engine at startup:
    Expect radar-hub: diagnose Job engine enabled with model_provider, model, namespace, and image. If it says no diagnose engine configured, hub.agent.enabled isn’t set on the running release.
  2. The sandbox resources exist:
  3. Start an investigation from a connected cluster in the web app. The first time anyone in your organization opens Diagnose, a disclosure card describes what leaves the cluster; one acceptance in the web app covers the whole organization. Until then, investigations started through an API token are refused with AI Diagnose needs your organization's acknowledgment, and a token can’t accept the disclosure itself. Each turn creates a Job in the sandbox namespace:

Troubleshooting

Upgrade fails with “needs postgres.bundled.auth.password set explicitly”

The release uses the bundled eval Postgres with a generated password. Follow Add the agent to an existing install and pass the existing password back in.

Upgrade fails with “podDSN is required with an external database”

The chart can’t read your Postgres Secret to rewrite its host. Set hub.agent.credentials.podDSN to a DSN reachable from the sandbox namespace, or put the whole credential set in hub.agent.credentials.existingSecret.

helm upgrade --wait ends with “context deadline exceeded”

With service.web.type=LoadBalancer, --wait also waits for the load balancer to receive an address. If it never does, Helm times out and marks the revision failed, even though the manifests were applied and the pods may be healthy. Check the Service events (kubectl -n radar-hub describe svc radar-hub-web), fix the load balancer, and run the upgrade again, or leave out --wait.
After a failed revision, Helm treats the last successful revision as the release’s current state. A later helm upgrade --reuse-values (or --reset-then-reuse-values) builds on that older revision’s values, so settings you added in the failed upgrade, such as hub.agent, are silently dropped from the new revision. The running pods can keep working for a while, which hides the problem until the next change.Re-run the upgrade from the values you meant to apply:
On EKS Auto Mode, Failed build model due to unable to resolve at least one subnet means no subnet is tagged for load balancers. Tag the public subnets with kubernetes.io/role/elb=1 (or the private ones with kubernetes.io/role/internal-elb=1 for an internal load balancer). Also check the load balancer scheme: Auto Mode load balancers are internal unless it carries service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing, and the chart doesn’t expose annotations on the web Service. For anything beyond a pilot, publish through an Ingress or Gateway instead.

A turn never starts or never reports

  • Image pull errors on the Job pod. Sandbox Jobs use the default ServiceAccount in the sandbox namespace; the chart’s top-level imagePullSecrets doesn’t apply to them. For a private mirror, set image.agentSandbox.repository, pre-create the sandbox namespace with a pull Secret attached to its default ServiceAccount, and set hub.agent.sandbox.create=false.
  • Model calls time out. The sandbox needs HTTPS egress to your provider. Behind an egress proxy or a private endpoint, add rules with hub.agent.sandbox.networkPolicy.extraEgress.
  • Database connection errors in the Job pod. The DSN in HUB_AGENT_DB_DSN must resolve from the sandbox namespace, and the NetworkPolicy must allow the database port (hub.agent.sandbox.networkPolicy.postgresPort).
  • Vertex permission errors. The service account needs roles/aiplatform.user on hub.agent.vertex.project, and the model must be enabled for that project in Model Garden. A newly granted role can keep returning Permission 'aiplatform.endpoints.predict' denied for a few minutes while IAM propagates; retry before changing anything.