RAG reduces hallucinations, but it doesn't stop them. A model can retrieve the right documents and still state something they never say. For example, it might name a software product's pricing tier that appears nowhere in the retrieved data. To a user comparing tools before a purchase, a confidently wrong price looks exactly like a correct one. This talk presents the grounding gate built into OMR Reviews' multi-agent AI Buying Assistant. Before an answer reaches the user, the gate extracts its claims, and an LLM judge checks each one against the retrieved data. The talk covers how claims are extracted and matched to sources, where LLM-as-a-judge verification holds up and where it fails, and the trade-offs between latency, cost and strictness. The gate is one part of a larger evaluation package that includes LLM-judged regression tests, automated QA on every merge to staging that checks tool calls and trajectories, and post-hoc issue detection on live traffic. The talk closes with how these layers work together.
Izabel Cvetkovic is an AI Engineer at OMR Reviews in Hamburg. There she builds the AI Buying Assistant, a multi-agent system based on hybrid RAG, Neo4j knowledge graphs and LangGraph. She also leads the AI Tooling & Automation stream, a cross-team initiative on Claude Code adoption, shared tooling and architectural standards. She holds a PhD in Information Systems on human-AI collaboration in knowledge work, and her technical focus is on the limits of RAG, knowledge graphs and LLM evaluation.
