Azure Document Intelligence to Qdrant: Chunking for RAG
The short answer: Azure Document Intelligence's prebuilt-layout gives you one of the best layout analyses money can buy; Qdrant gives you excellent hybrid vector search. Wired together naively — flatten, split, embed — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy. The missing link is POMA: PrimeCut().ingest() consumes the raw analyze result (markdown mode or JSON mode, auto-detected), emits hierarchy-preserving chunksets, and PomaQdrant.upsert_poma_points() writes them as hybrid dense+sparse points that carry the chunkset's lineage in the payload. Retrieval comes back as assembled, prompt-ready cheatsheets.
What each end of the pipeline actually provides
Azure Document Intelligence (prebuilt-layout) returns its analysis in two shapes: markdown mode (outputContentFormat="markdown") — one reading-order markdown string with inline HTML tables, <!-- PageBreak --> delimiters, and page-furniture comments — or classic JSON mode, a paragraphs[] array ordered by span offset with roles like title and sectionHeading, plus tables[] as cell grids. Multi-column de-interleaving happens server-side. What it doesn't provide: cross-page hierarchy (a sectionHeading on page 41 has no link to the title on page 3), retrieval units, or figure bytes. Details: The Optimal Chunker for Azure Document Intelligence.
Qdrant provides named dense + sparse vectors on one point, payload indexes for filtered ANN, and HNSW with quantization options. What it doesn't provide: any opinion about what a point should contain. It retrieves nearest neighbors of whatever you embedded — including context-free fragments, if that's what you gave it. Details: The Optimal Chunks for the Best Retrieval in Qdrant.
The naive wiring, and where it breaks
The pair-specific failure here is well documented, because Microsoft ships it: the RAG samples and accelerators in Azure's own ecosystem take prebuilt-layout's markdown output and run fixed-size or token-window splitters over it. Everything the layout model recovered vanishes before Qdrant sees a single vector:
PageBreakdelimiters are flattened away, so no Qdrant payload can cite a page. The one field that makes a legal or compliance answer auditable — "clause 14.2, page 41" — is gone at ingestion, not at retrieval.- Role-tagged headings are treated as ordinary text.
titleandsectionHeadingparagraphs — the skeleton of the document — get cut into whatever 512-token window they happen to fall in. Retrieved fragments arrive in the prompt without lineage, and the LLM answers out of context. - HTML tables are sliced mid-row, separating header rows from data rows, which is precisely the failure Azure's table extraction was supposed to prevent.
- Splitter overlap inflates the collection. Every boundary span is embedded twice, which in Qdrant means a larger HNSW graph and top-k results where hits 2 and 3 are near-duplicates of hit 1, crowding out the passage that actually answers the question.
The result: an expensive, excellent analysis reduced to the same context-free fragments a plain-text scrape would have produced. Qdrant executes the retrieval flawlessly — over inputs that were ruined one step earlier.
The pipeline, end to end
pip install azure-ai-documentintelligence 'poma[qdrant]'import json
import os
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut
from poma.integrations.qdrant import PomaQdrant
# 1. Azure Document Intelligence — your existing call, unchanged.
di = DocumentIntelligenceClient(
endpoint=os.environ["AZURE_DI_ENDPOINT"],
credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
poller = di.begin_analyze_document(
"prebuilt-layout",
body=f,
output_content_format="markdown", # omit for classic JSON mode — POMA handles both
)
analysis = poller.result()
with open("contract.azure-di.json", "w") as f:
json.dump(analysis.as_dict(), f)
# 2. The missing link — raw analyze result in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.azure-di.json", # Azure shape auto-detected
download_dir="store",
filename="contract.poma",
)
# 3. Qdrant — hybrid points with hierarchy payloads, one call.
qdrant = PomaQdrant(
url=os.environ["QDRANT_URL"],
api_key=os.environ["QDRANT_API_KEY"],
cloud_inference=False, # True needs Qdrant Cloud's inference service; False embeds locally via fastembed and works on OSS/local Qdrant too
collection_name="contracts",
dense_model="sentence-transformers/all-MiniLM-L6-v2",
sparse_model="Qdrant/bm25",
dense_size=384,
auto_create_collection=True,
)
qdrant.upsert_poma_points(result)
# 4. Retrieve prompt-ready context.
cheatsheets = qdrant.get_cheatsheets(
query="What are the early termination conditions?",
limit=3,
)
print(cheatsheets[0]["content"])POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately — never a silently degraded collection). Markdown mode is split on PageBreak comments with PageHeader/PageFooter/PageNumber furniture dropped; JSON mode is reconstructed from paragraphs[] in span-offset order with roles mapped to heading levels and tables[] converted to HTML. Figures — which Azure never returns as bytes — are counted as offloaded in content_metadata, so the loss is visible, never silent. To force or suppress detection, set external_ocr_source to "azure_di" or "none".
Measured on our reference legal-document benchmark, this pipeline answers the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss — methodology in Document Ingestion & Chunking for RAG.
Metadata mapping: POMA fields → Qdrant primitives
One point per chunkset. upsert_poma_points writes exactly this payload: {"chunkset_index", "chunks", "file_id", "text"} plus "chunk_details" (the chunkset's chunk records — chunk_index, content, depth, file_id, code) while store_chunk_details stays at its default True.
| POMA field | Qdrant primitive | What it enables |
|---|---|---|
to_embed | dense vector + BM25 sparse vector (named vectors, one point) | paraphrase + exact-term hybrid retrieval |
text (a copy of to_embed) | payload field | read back what was embedded |
file_id | payload field (no index is created for you — add one with create_payload_index) | scope queries to one document |
chunkset_index | payload field | identifies the point: the id is a UUIDv5 of file_id + chunkset_index |
chunks (chunk indices) | payload field | joins the point back to its chunk records |
chunk lineage (chunk_index, content, depth, code) | payload (chunk_details) | cheatsheet assembly without a second store |
page (from PageBreak delimiters or span-to-page mapping) | not written by PomaQdrant — the SDK's PomaChunk drops it; it survives on the archive's chunk records | page-cited answers, page-range filters — see below |
get_cheatsheets reads only that payload back: it groups each hit's chunkset_index + chunks by file_id and assembles the text from the hit's chunk_details (or from a chunk_data you pass instead).
Page numbers are in the .poma archive, which the ingest call above already wrote to store/contract.poma. Attach them to the same points:
from poma.utils import unpack_poma_archive
from poma.integrations.qdrant.qdrant_poma_utils import chunk_uuid_string
page_of = {c["chunk_index"]: c.get("page") for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]}
for cs in result.chunksets:
leaf = cs.chunks[-1] # chunk indices are in document order; the last one is the leaf
qdrant.set_payload(collection_name="contracts", payload={"page": page_of[leaf]}, points=[chunk_uuid_string(cs.file_id, cs.chunkset_index)])Frequently asked questions
How do I get Azure Document Intelligence results into Qdrant for RAG?
Run prebuilt-layout as you already do — either mode — and save the raw analyze result. PrimeCut().ingest() auto-detects the Azure shape and rebuilds the cross-page hierarchy into chunks and chunksets; PomaQdrant.upsert_poma_points(result) writes hybrid points whose payload carries file_id, chunkset_index, the chunkset's chunk indices, the embedded text, and chunk_details (chunk_index, content, depth) for assembly. Retrieve with get_cheatsheets(query=...).
Does POMA handle both prebuilt-layout markdown mode and JSON mode on the way to Qdrant?
Yes. Markdown mode is split on PageBreak comments (furniture comments dropped); JSON mode is reconstructed from paragraphs[] in span-offset order, roles mapped to headings, cell grids converted to HTML. Both end in identical chunks and chunksets — the Qdrant collection looks the same either way.
What Qdrant payload fields should Azure Document Intelligence chunks carry?
PomaQdrant writes file_id, chunkset_index, chunks, text and chunk_details (the chunk records with chunk_index, content and depth) by default. It does not write page, and it creates no payload indexes: add a payload index on file_id yourself for per-document scoping, and if you need page-cited answers or page-range filters, read page (recovered from PageBreak delimiters or the span-to-page mapping) from the archive's chunk records with unpack_poma_archive and attach it with client.set_payload on the same point ids.
Do Azure Document Intelligence tables survive the trip to Qdrant?
Yes — inline HTML tables (markdown mode) and reconstructed cell grids (JSON mode) enter the chunk layer as HTML, merged cells intact, never cut mid-row, and each is embedded with its heading lineage. Fixed-size splitters routinely separate header rows from data rows instead.
Why do Microsoft's own RAG samples underperform when the vectors land in Qdrant?
They run fixed-size or token-window splitters over the markdown output, so page breaks and role-tagged headings vanish before Qdrant sees a vector: no page number left to attach, no hierarchy, tables cut mid-row, near-duplicate overlap points. Qdrant retrieves exactly what it was given — context-free fragments.
Related recipes
Same parser, different store: Azure Document Intelligence → Pinecone · Azure Document Intelligence → Weaviate · Azure Document Intelligence → pgvector
Same store, different parser: Mistral OCR → Qdrant · LlamaParse → Qdrant · Textract → Qdrant
Foundations: The Optimal Chunker for Azure Document Intelligence · The Optimal Chunks for Qdrant · All pipeline recipes · RAG chunking guide