LLM inference has quietly become the biggest cost line in most AI products. As models get bigger and traffic grows, teams are watching their bills scale linearly while their margins collapse. The teams that win in 2026 won't be the ones with the biggest models. They'll be the ones who figured out how to serve them 10x cheaper without sacrificing quality. This talk goes deep on the engineering frontier of LLM inference, covering the levers that actually move the needle in production: KV-cache optimization, speculative decoding, quantization (INT8, INT4, GPTQ, AWQ), continuous batching, PagedAttention, and speculative parallelism. We'll cover the trade-offs between vLLM, TensorRT-LLM, and custom serving stacks, and when each one makes sense. Then we'll get into the architectural decisions that matter most: intelligent model routing, hybrid hosted-vs-self-hosted deployments, caching strategies for LLM workloads, and how to design cost-aware inference pipelines that degrade gracefully under load. Expect production war stories, real benchmarks, and a concrete framework for cutting inference costs without cutting corners. No hype, no vendor pitches, just the engineering choices that separate teams burning cash from teams shipping AI profitably at scale.
Devang Sharma is a Senior AI Engineer at Meta with over 8 years of experience building distributed systems and AI infrastructure at scale across Meta, Amazon, and Cisco. Throughout his career, he has shipped high-impact engineering solutions, including a core WhatsApp feature serving over 2 billion users, and led large-scale infrastructure optimization initiatives saving over $120,000 monthly. Beyond his corporate work, Devang is the Founder of TechElite India and an active global mentor who has guided more than 20,000 engineers through platforms such as Exponent, Topmate, Interviewing.io, and Tutort Academy.
