Case study
Agentic document intelligence, from PDFs to structured data and answers
Docling parsing and LLM extraction turn PDFs, forms, and reports into JSON, a knowledge graph, and grounded QA, at under 3s per page.
- Role
- AI / ML Engineer
- Context
- Zyoin · Acranton · projects
- Core stack
- Docling, LLMs, Gemini API
- ingestion per page
- <3s
- grounded QA response
- 1.5s
- healthcare form throughput
- +30%
Problem
Most enterprise knowledge sits in documents nobody can query: PDFs, scanned forms, and long reports. The goal was to make that content usable in two ways at once, as structured data that downstream systems can consume and as answers people can get by asking questions.
Constraints
- Messy inputs. Layouts, tables, forms, and handwriting vary widely between documents.
- Throughput. Collections are enterprise-scale, so ingestion has to stay fast per page.
- Grounded answers. Answers and summaries must come from the documents themselves.
- Healthcare data. Some of it was medical. Extraction had to be fast enough for clinicians to use.
Approach
Parse structure before reasoning
Each document goes through Docling first, which recovers layout, tables, and text. The LLM then works on structured content instead of raw PDF bytes. That is cheaper and more reliable than asking the model to also work out the page layout.
Agent-driven extraction and classification
An extraction agent classifies each document and extracts fields into structured JSON and a knowledge graph. The same content is chunked and embedded into a vector index for semantic search.
Conversational QA and summarization
A RAG pipeline over the vector index answers questions and summarizes across ingested documents, with 1.5s grounded QA latency on enterprise-scale collections.
The same pattern in healthcare
- Form extraction (Acranton). A Gemini API–based extractor converts unstructured healthcare documents into structured data in under 3 seconds, raising throughput 30%.
- Handwritten prescriptions. Gemini Flash converts handwritten prescriptions into JSON within 5 seconds, making the data accessible to clinicians.
Results
- document ingestion per page
- <3s
- grounded QA response latency
- 1.5s
- healthcare form → structured data; throughput gain
- <3s · +30%
- handwritten prescription → JSON
- <5s
Stack
- Parsing
- Docling
- Models
- LLMs
- Gemini API
- Gemini Flash
- Serving
- FastAPI
- Vector DB
- RAG
Related writing
- Retrieval-Augmented Generation (RAG): the QA side of this pipeline.
- Search NeurIPS-2022 Similar Papers: semantic search over a document collection with sentence embeddings.