Abstract
Every conversation with an AI starts from zero. To help users, we have to populate an LLM's context with correct information, in the right amount, at tolerable latency.
We explore:
- Populating the context: static knowledge injection, RAG and the agentic loop.
- Types of information — and which database each one needs.
- Dealing with stale context in vector knowledge bases: decay and priority metrics on the reranking phase — or pushed inside the HNSW KNN walk itself.
- Bypassing embedding context limitations with semantic centroids.
This is an engineering talk with learnings from different use cases, from local chatbots to large e-commerce applications. Forgetting is not a bug. It's a design choice.
Bio
More than 13 years of software development experience. Software Architect, specializing in AI services (RAG, model self-hosting), cloud solutions (AWS, Hetzner), and infrastructure optimization. + 6 years at moebel.de. Certifications:
- TOGAF Enterprise Architecture Foundation
- AWS Certified Solutions Architect – Associate
- AWS Certified Developer – Associate
- AWS Certified ML Engineer – Associate Education:
- K.T.N in Logistics (Technical Sciences)
- Retrained in software development (second diploma)