If you know how to run production applications, you already know more about LLM inference than you might think — until someone mentions KV caches, tensor parallelism, continuous batching, or prefill/decode and suddenly the stack looks completely different. This session maps familiar platform engineering concepts like scheduling, scaling, routing, observability, and resource management to the LLM inference world, while explaining the concepts that really are new. We’ll build and operate an LLM inference service along the way, leaving platform engineers with the mental model and vocabulary they need to start operating AI workloads confidently.
Annie Talvasto is an award-winning international technology speaker and leader who has spoken at 100+ events worldwide, including KubeCon + CloudNativeCon and Microsoft Build & Ignite. She has been recognized with the CNCF Ambassador, Azure & AI Platform MVP awards. She co-founded Kubernetes & CNCF Finland in 2017 and has served as a KubeCon track chair, Secure AI Summit Program Chair, and host of CNCF’s Cloud Native Live.
