A governed multi-agent assistant for enterprise workflows

Problem

Business teams needed one assistant that could find information, reason over it, and complete tasks across their workflows, working on enterprise data that carries access controls and compliance obligations.

A single prompt-and-retrieve chatbot couldn’t do this reliably. Different steps need different tools and different data permissions, and a compliance team needs to be able to reconstruct what the system did and why.

Constraints

• Latency. Interactive use, so responses had to stay fast while many sessions ran concurrently. The system holds a sub-800ms average.
• Multi-tenancy. Concurrent sessions from multiple tenants on shared infrastructure, so it had to scale horizontally.
• Governed data. Every agent-facing data source needed lineage, access control, and compliance coverage.
• Auditability. Model versions, prompts, and responses all had to be reviewable after the fact.
• Release safety. Web and application layers ship together, so builds, tests, and deploys had to be automated.

Approach

Specialized agents behind one orchestrator

I architected the assistant in LangGraph as a set of specialized agents (retrieval, reasoning, and task execution) coordinated by an orchestration graph. Splitting by responsibility keeps each agent’s prompt and toolset small. That makes failures easier to localize, and each agent can be tested and improved on its own.

Structured data through Genie

Many questions are really queries over structured tables. For those I built Genie-based conversational agents on Databricks that translate natural language into queries over governed enterprise data. This cut analyst query turnaround by 30%.

Governance as part of the architecture

Governance was designed in, not bolted on:

• Unity Catalog catalogs, schemas, and governed tables for every agent-facing source, enforcing lineage and access control.
• Guardrails and audit logging in the request path, with prompt and response auditing on every turn.
• Model versioning and access policies, so any answer can be traced to the model and configuration that produced it.

These controls improved compliance readiness and cut incident response time by 18%. When something goes wrong, the audit trail shows which agent, model version, and data source were involved.

Horizontal scale on Databricks

Agents run on Databricks Model Serving, with job clusters for heavier work and automated retraining pipelines. This cut deployment overhead by 25%. The system scales horizontally across concurrent multi-tenant sessions while holding a sub-800ms average response time.

Shipping safely

Azure DevOps pipelines build, test, and deploy both the web and application layers. Release cycle time fell 20% and deployment failures fell 15%.

Results

average response latency, concurrent multi-tenant sessions

<800ms

end-to-end task completion time

−22%

analyst query turnaround (Genie agents)

−30%

deployment overhead (Model Serving + job clusters)

−25%

release cycle time / deployment failures

−20% / −15%

incident response time

−18%

Stack

Agents

• LangGraph
• Databricks Genie
• Guardrails
• Audit logging

Platform

• Databricks Model Serving
• Job clusters
• Unity Catalog

Delivery

• Azure DevOps CI/CD
• Model versioning
• Automated retraining