Ravikumar M. N.
Menu
Résumé
Selected work

Case study

An enterprise GenAI copilot with agentic RAG and tool use

One copilot over documents, tables, and analytics tools, with hybrid retrieval and query planning, answering in 1.8s end to end.

Context
Project
Core stack
LangChain, LlamaIndex, Azure
end-to-end response latency
1.8s
vector search latency at scale
<120ms
structured and unstructured sources, one copilot
SQL + RAG

Problem

Enterprise knowledge was split between unstructured documents and structured tables. People needed one place to ask questions that span both: analytics questions, document questions, and questions that need a bit of each. The answers had to be grounded in the source data.

Constraints

  • Mixed sources. Some questions need semantic recall over documents, some need exact keyword matches, and some need SQL over tables.
  • Grounding. Answers had to be traceable to retrieved evidence, not invented by the model.
  • Interactive latency. A copilot people actually use has to answer in seconds, with retrieval taking only a small part of that time.
  • Operability. Releases needed tracking, evaluation, and versioning so a quality regression could be caught and rolled back.

Approach

Plan first, then retrieve

Multi-agent orchestration splits each request into three steps: query planning, data retrieval, and response synthesis. The planner decides which route a question needs: hybrid retrieval, a SQL agent for structured data, or tool calls for analytics and document workflows.

Hybrid retrieval across structured and unstructured data

Vector search alone misses exact identifiers and rare terms, and keyword search alone misses paraphrases. I combined vector and keyword retrieval across structured and unstructured sources, which improved answer relevance and grounding accuracy. The two result sets are merged and ranked before synthesis.

Synthesis with memory and guardrails

The synthesis step keeps contextual memory across turns and applies guardrails before an answer is returned.

Latency work

I optimized both the inference path and the retrieval path, reaching 1.8s end-to-end response latency with vector search under 120ms at scale.

Agentic RAG copilotA query planner decomposes each question and routes it to hybrid retrieval (vector and keyword search, merged and ranked), a SQL agent for structured data, and tool calls. A synthesis step with memory and guardrails produces a grounded answer in about 1.8 seconds end to end. MLflow tracks, evaluates, and versions releases on Azure and Databricks.HYBRID RETRIEVALUser questionQuery plannerdecompose · routeVector searchdense · <120msKeyword searchlexical matchSQL agentstructured dataTool callsanalytics · documentsMerge & rankfused top-kResponse synthesismemory · guardrailsGrounded answer1.8s end-to-endMLflow: tracking · evaluation · versioned releases on Azure + Databricks
Fig.Routing and retrieval paths. The merge-and-rank step is simplified.

Shipping

The system is deployed on Azure and Databricks, with MLflow for model tracking, evaluation, and versioned releases.

Results

end-to-end response latency
1.8s
vector search latency at scale
<120ms
vector + keyword retrieval, structured + unstructured
Hybrid

Stack

Orchestration
  • LangChain
  • LlamaIndex
  • SQL agents
  • Tool calling
Retrieval
  • Vector search
  • Keyword search
  • Hybrid retrieval
Platform
  • Azure
  • Databricks
  • MLflow