Support teams need incoming issues sorted into the right category so they reach the right people. But what happens when the categories aren't a flat list when they're three levels deep, and the valid options at each level depend on the choices above them? This talk walks through a real production problem: automatically classifying support cases into a three-tier hierarchy. The obvious answer, train a classification model but it turned out to be a poor fit. Hierarchical classification is genuinely hard for traditional models: early mistakes cascade down the tree, rare categories have too few examples to learn from, and off-the-shelf tools don't understand which options are valid under which parent. Instead of training a model, we reframed the problem as retrieval. By building a repository of common issues for each leaf category, embedding it, and using a hybrid RAG search to match new cases against it, we let semantic similarity do the classification work as a result no model training required. I'll cover why traditional approaches struggle with hierarchical problems, how the retrieval approach sidesteps those issues, the tradeoffs we accepted, and where this pattern does (and doesn't) make sense.
Vaibhav is a software engineer based in the Bay Area with more than eight years of experience in the customer experience, customer support, and CRM domain. He works on applied AI and LLM-powered agents, has built and shipped production AI agent systems, and enjoys finding practical ways to solve real engineering problems with retrieval, embeddings, and agentic patterns.
