Most teams secure LLM applications the way they secure software: test before launch, add a guardrail, move on. That model breaks down here. Models get updated, prompts change, tools get added, and an agent that was safe last quarter has a different attack surface this quarter. This talk covers three gaps I keep running into in production AI systems, and what to do about each. First, why red teaming a model has to be continuous rather than a pre-launch gate, including what changed across iterations while testing NVIDIA's Nemotron content safety filter model with their AI engineering team. Second, guardrails. What content filters and safety classifiers actually catch, the failure modes teams assume are covered but aren't, and how to build an evaluation loop that tells you when a guardrail has quietly stopped working. Third, identity, the gap I think is most underestimated. Agents authenticate, hold credentials, call tools, and act on behalf of users, but the identity infrastructure we point at them was designed for humans making individual decisions. Sessions, MFA, least privilege, and audit trails don't map cleanly onto a non-deterministic system that spawns sub-agents and chains tool calls across systems. I'll walk through where that mismatch creates real exposure and what agent-native identity has to look like instead. Attendees will leave with a working model for continuously testing their own LLM and agent deployments, not a threat list.
Sai Varma is an AI/LLM Security Researcher at Votal AI, where he focuses on adversarial testing of language models and agentic systems. He co-built Votal's AI red teaming framework alongside the company's CTO and red-teamed NVIDIA's Nemotron content safety filter model, working directly with NVIDIA's AI engineers to address the security issues surfaced during testing.
