Skip to main content

Deploy on Google Kubernetes Engine

The Helm chart for GKE: a scalable gateway tier, one admin pod, a migration Job before every rollout, Cloud SQL through the Auth Proxy with IAM auth, Secret Manager as a file, GCS for files and Pod Security restricted.

On this page

The repository ships a Helm chart, deploy/helm/systemprompt-dynapps/, that runs the same image, profile and secrets.json as the one-node deployment on GKE. The chart is new. It lints and renders cleanly and its profile validates against the profile schema, but it has not yet been installed on a real GKE project. The operator runbook, with every command, is docs/install/gke.md in the repository.

Topology

piece what it runs how many
Gateway Deployment server.role: gateway: /v1 inference, the bridge, OAuth, well-known and health routes. No scheduler, MCP or agent workers. 3 to 12, scaled by a HorizontalPodAutoscaler on CPU, with a PodDisruptionBudget
Admin Deployment server.role: admin: the console, the scheduler, MCP servers and agents exactly 1, Recreate strategy, because two would run the scheduler twice
Migrate Job the image's one-shot migrate step: schema migrations, then the bootstrap admin once, as a Helm pre-install and pre-upgrade hook

The load balancer sends /v1 to the gateway pods and everything else to the admin pod. Pods never migrate on boot and refuse a schema that is behind, so the Job always runs before any new pod starts.

What each pod carries

  • Database. A Cloud SQL Auth Proxy runs as a native sidecar with IAM database authentication over private IP. The server connects to 127.0.0.1:5432; the proxy handles TLS and identity. Cloud SQL is PostgreSQL 16 in HA mode, and replica reads stay off.
  • Secrets. Google Secret Manager holds the whole secrets.json. The GKE Secret Manager CSI add-on mounts it as a file, and it is read once at boot. Rotation is a new secret version and a restart.
  • Files. Uploaded and generated files live in a Cloud Storage bucket every pod shares, reached through Workload Identity.
  • Health. Startup and liveness probes use /livez. Readiness and the load balancer health check use /readyz, which answers 503 with {"status":"saturated"} when the pod is over server.max_in_flight, so a saturated pod leaves rotation instead of queueing.
  • Timeouts. The backend timeout is 600 seconds, long enough for streamed completions.
  • Metrics. A separate /metrics listener on server.metrics_port, with optional Managed Service for Prometheus scraping.
  • Security. Every container meets Pod Security restricted: uid 1000, read-only root filesystem, all capabilities dropped, RuntimeDefault seccomp, no privilege escalation. Each writable path is an emptyDir, and /tmp is memory-backed.
  • Network. A default-deny NetworkPolicy allows DNS, the metadata server, the Cloud SQL private range and the Private Google Access ranges. Providers outside Google need an explicit egress rule.

Before installing

Create these in the Google Cloud project first:

  1. A GKE cluster, version 1.29 or newer, with Workload Identity, Dataplane V2 and the Secret Manager add-on.
  2. A Cloud SQL for PostgreSQL 16 HA instance on private IP, with IAM authentication and an IAM database user for the service account.
  3. A Cloud Storage bucket for files.
  4. A Secret Manager secret holding secrets.json.
  5. An Artifact Registry repository holding the mirrored, verified image.
  6. One Google service account with Cloud SQL client and instance user, Secret Manager accessor, and Storage object admin roles. Bind it to both of the chart's Kubernetes service accounts, the app's and the migrate Job's.

What stays on the pod

The gateway accounting journal, exports, the rendered public site, caches and logs live in per-pod emptyDir volumes. A replaced pod loses any journal entries not yet written to Postgres. Everything else is rebuilt at boot.

Rolling back

A Helm rollback is safe only while the schema has not moved forward. The migrate hook does not run on rollback, and a pod refuses a schema newer than it knows. Take a Cloud SQL backup before every upgrade, and restore it before rolling back across a release that added migrations.