01 What happened
At KubeCon + CloudNativeCon North America 2026, Google Cloud is introducing and expanding a suite of open-source AI-native components for Kubernetes and Google Kubernetes Engine. New CNCF Sandbox projects include OpenRL, a Kubernetes-native RL post-training service that maximizes accelerator utilization; llm-d 1.0, a distributed inference control plane co-founded with Red Hat, IBM Research, CoreWeave, and NVIDIA; and Agent Substrate, an orchestration runtime for multi-turn AI agents. Google Cloud continues core contributions to gVisor, a kernel-level container sandbox. The company reported testing Gemma 4 on the Legal Agent Benchmark by Harvey.ai: standard SFT accuracy plateaued at roughly 50%, while OpenRL RL loops lifted accuracy to 70%. In production deployments, llm-d's prefix-cache-aware routing doubled KV cache hit rates from 35% to 70% and cut median time-to-first-token by 70–85% on multi-turn agent workloads.
02 Key details
- Google Cloud at KubeCon 2026 introduced OpenRL (CNCF Sandbox), a Kubernetes-native RL post-training service using cooperative accelerator time-slicing and delta-weight sync.
- llm-d 1.0 (CNCF Sandbox), co-founded by Google Cloud, Red Hat, IBM Research, CoreWeave and NVIDIA, provides prefix-cache-aware routing and 3-tier KV cache offload.
- Agent Substrate (CNCF Sandbox) was launched as an open orchestration runtime for multi-turn AI agents; Google Cloud continues core contributions to gVisor for kernel-level sandboxing.
- Testing Gemma 4 on the Legal Agent Benchmark (Harvey.ai) resulted in a gain of accuracy from ~50% (SFT) to 70% with OpenRL RL loops.
03 Why it matters
Google Cloud consolidates RL post-training, distributed inference, and agent safety in a single open Kubernetes stack, promising higher accelerator utilization and lower latency without vendor lock-in.
04 Who it matters to
Platform Engineers, artificial intelligence researchers, AI agent developers, devOps specialists and IT architects.