As organizations move beyond AI proofs of concept into production deployments, a new set of challenges emerges. The question is no longer whether AI agents can work, but whether they can be trusted to operate reliably at scale. This talk examines the critical gap between building an agent and running one in real-world enterprise environments.
We reframe reliability as a multi-dimensional problem spanning infrastructure, model availability, output validity, semantic correctness, and behavioral consistency. Unlike traditional software systems, AI agents are inherently probabilistic, requiring new approaches to service-level expectations, including accuracy thresholds, guardrails, and human-in-the-loop workflows.
The session explores practical patterns for designing production-ready agents, including deterministic orchestration layered with controlled model autonomy, structured outputs with validation, and multi-model fallback strategies. It also highlights the importance of observability beyond uptime—covering traceability, quality evaluation, cost monitoring, and behavioral analytics—to detect and mitigate silent failures.
Finally, we address system integration concerns such as policy enforcement, auditability, data governance, and safe rollout strategies. Attendees will leave with a concrete mental model for architecting agents as composable, governable systems rather than monolithic LLM calls, along with actionable guidance on building evaluation-first workflows that enable continuous improvement and safe scaling.
This talk is designed for engineering and platform teams tasked with turning AI agents into dependable, enterprise-grade systems.
Harshada Jivane is a Senior Machine Learning Engineer at Laurel, specializing in GenAI, LLMs, and large-scale distributed systems. She has previously built AI and document understanding systems at Intuit and Hewlett Packard Enterprise, with a focus on production-grade machine learning and real-world applications.
With a background in big data, machine learning, and RAG-based systems, Harshada works at the intersection of AI research and engineering, turning complex models into scalable, practical solutions.
