LLMs on Kubernetes: Same Cluster, Different Threat Model

LLMs are showing up in Kubernetes clusters everywhere, but they're not like other workloads. They take untrusted user input and decide what to do with it. That's a security problem Kubernetes doesn't solve for you.

In this session, we'll look at what the OWASP LLM Top 10 means for Kubernetes operators: prompt injection, data leakage, supply chain risks, and over-permissioned tool access. Then we'll build a solution, an LLM gateway that handles policy enforcement so your model runtime doesn't have to.

We'll use mirrord to develop and test policies locally against real cluster dependencies, no image rebuilds, no redeploys. We'll also use Cloudsmith to show how to store and govern model artifacts with the same supply chain controls you use for container images. Live demo included.

You'll leave with a mental model for LLM security and a workflow you can actually use.