Azure Document Intelligence Chunking: Layout Output to RAG
Azure Document Intelligence chunking is the step Azure leaves to you. The prebuilt-layout model returns an excellent analysis — reading order, paragraph roles, HTML tables, a section tree — but an analysis is not a set of retrieval units. Something downstream has to decide what one embedding vector represents, and that decision, not extraction accuracy, is where most Azure Document Intelligence RAG pipelines lose quality.
This page covers three things: the quickest path from a prebuilt-layout result to embeddable chunks, Microsoft's own recommended semantic chunking recipe and where its ceiling is, and a full pipeline from begin_analyze_document to retrieval.
Quick start: prebuilt-layout output to chunks
Run the analysis exactly as you do today, save the raw result, hand it to PrimeCut. The Azure shape is auto-detected, so there is no adapter to write.
import json
import os
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.ai.documentintelligence.models import DocumentContentFormat
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut
# 1. Your existing Azure Document Intelligence call, unchanged.
di = DocumentIntelligenceClient(
endpoint=os.environ["AZURE_DI_ENDPOINT"],
credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
poller = di.begin_analyze_document(
"prebuilt-layout",
body=f,
output_content_format=DocumentContentFormat.MARKDOWN, # omit for classic JSON mode
)
result = poller.result()
with open("contract.azure-di.json", "w") as f:
json.dump(result.as_dict(), f)
# 2. Chunk the saved analyzeResult. Azure's OCR is not re-run.
poma = PrimeCut() # reads POMA_API_KEY
chunked = poma.ingest("contract.azure-di.json")
print(len(chunked.chunks), len(chunked.chunksets))
print(chunked.chunksets[0].to_embed) # embedding-ready text, heading path includedBoth output modes work. output_content_format=DocumentContentFormat.MARKDOWN gives one reading-order markdown string; omitting it gives the classic analyzeResult JSON with paragraphs[] and tables[]. PrimeCut detects which one it received and normalizes both to the same chunks and chunksets, so a pipeline that switches modes later does not change its index shape.
What the layout model returns, and where it stops
The layout model of Azure AI Document Intelligence is one of the strongest table and structure extractors available, and it is worth being precise about what lands in the response.
Markdown mode. outputContentFormat=markdown puts the whole document into content as a single reading-order markdown string. Since v4.0 2024-11-30 (GA), tables are rendered as HTML tables so merged cells and multirow headers survive, and selection marks use the Unicode characters ☒ and ☐. Page furniture arrives as HTML comments:
# Annual Report 2025
<!-- PageHeader="Contoso Ltd. — Confidential" -->
## Financial highlights
<table><tr><th>Quarter</th><th>Revenue</th></tr><tr><td>Q1</td><td>4.1M</td></tr></table>
<!-- PageNumber="1" -->
<!-- PageBreak -->JSON mode. The default response is structure as data, not prose:
paragraphs[]— each withcontent,spans(offset,lengthinto the top-level content) andboundingRegions(carryingpageNumber). An optionalroleis one oftitle,sectionHeading,footnote,pageHeader,pageFooter,pageNumber,formulaBlock.tables[]—rowCount,columnCount, andcells[]withrowIndex,columnIndex,rowSpan,columnSpanand akind(columnHeader,rowHeader,stubHead,description,content).sections[]— a hierarchy where "the hierarchical structure is maintained inelementsfor each section". The entries are JSON pointers:"/paragraphs/0","/sections/1","/sections/2".figures[]—id,boundingRegions,spans,elementsand an optionalcaption.pages[]—pageNumber, pluslines,wordsandselectionMarks.
Two things Azure does exceptionally well: tables, and multi-column reading order. De-interleaving happens server-side, so span offsets in JSON mode and the content string in markdown mode already follow the order a human reads.
Three things it deliberately does not do:
- Chunking. The response is an analysis. Nothing in it says where a retrieval unit begins or ends.
- Resolved hierarchy on the text.
sections[]really is a tree, but it is a tree of pointers next to the text, not attached to it. AsectionHeadingparagraph carries no field naming the chapter it belongs to. To use the tree you resolve every/paragraphs/Npointer yourself, recursively, and rebuild the path — and none of it survives into markdown mode, where the tree collapses into#levels. - Figure pixels in the result. Figures are regions with captions. Cropped images exist only when you pass
output=figures, and then they are a separate fetch from/analyzeResults/{resultId}/figures/{figureId}— never bytes inside the JSON you saved.
Microsoft's recommended semantic chunking, and its ceiling
Microsoft's RAG guidance for Document Intelligence is explicit and, as far as it goes, correct: prefer semantic chunking over fixed-size chunking, and use the layout model's markdown output as the substrate. Fixed-size windows, the docs note, "can result in severing words, sentences, or paragraphs". The recommended recipe splits that markdown on its headers, using the LangChain document loader:
from langchain_community.document_loaders import AzureAIDocumentIntelligenceLoader
from langchain.text_splitter import MarkdownHeaderTextSplitter
loader = AzureAIDocumentIntelligenceLoader(
file_path="contract.pdf",
api_key=key,
api_endpoint=endpoint,
api_model="prebuilt-layout",
)
docs = loader.load()
headers_to_split_on = [("#", "Header 1"), ("##", "Header 2"), ("###", "Header 3")]
text_splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers_to_split_on)
splits = text_splitter.split_text(docs[0].page_content)That from langchain.text_splitter import ... line is Microsoft's sample as published; on current LangChain the splitters moved out of the langchain package, so import from langchain_text_splitters instead.
This is a real improvement over a character splitter, and if you are starting from zero it is the right first move. Its ceiling shows up at retrieval time.
MarkdownHeaderTextSplitter gives each split a metadata dict with the headers above it, which sounds like hierarchy. Two problems follow. First, the metadata is not in page_content, so unless you concatenate it by hand the embedded text is still an orphan fragment: the paragraph under ### Early termination is vectorized without the words "Master Services Agreement" or "Termination clauses" anywhere in it, and it will not match a query phrased in those terms. Second, a section is not a chunk. A three-page ## Indemnification section becomes one enormous split that dilutes into an unusable vector; a two-line ### Notices becomes a tiny one. You end up bolting a RecursiveCharacterTextSplitter onto the output to cut the long ones, and that second pass re-introduces exactly the mid-table, mid-sentence cuts semantic chunking was meant to avoid.
Everything else in the analyzeResult is gone by then too. Page numbers went with the <!-- PageBreak --> comments, so no answer can cite "page 41". PageHeader and PageFooter furniture stayed in the text and gets embedded on every page as noise. The sections[] tree was never consulted.
For the general taxonomy of these failure modes, see the RAG chunking strategies guide.
Azure Document Intelligence chunking on the raw analyzeResult
POMA's bring-your-own-OCR path exists for teams who already run a parser and are not going to swap it — and enterprise Azure shops rarely get to swap out Document Intelligence. You upload the unmodified result and only POMA's downstream stages run on it. Concretely, for azure_di:
- Shape validation up front. The payload is fingerprinted on the
analyzeResultenvelope, or onparagraphs[]carrying Azure'sspansshape. A mislabeled or corrupted upload fails with a 422, never a quietly degraded index. - Markdown mode. Detected on
contentFormat: "markdown".contentis split on<!-- PageBreak -->, which restores real page anchoring, and thePageHeader,PageFooterandPageNumbercomments are removed rather than embedded. - JSON mode. Paragraphs are ordered by
spans[0].offset— reading order, multi-column included, straight from Azure, no geometry re-sorting.role: "title"maps to#androle: "sectionHeading"to##.pageHeader,pageFooterandpageNumberparagraphs are routed to metadata instead of the content flow. Each element's page comes fromboundingRegions[0].pageNumber. - Tables reconstructed whole.
cells[]are laid back onto arowCount×columnCountgrid,rowSpanandcolumnSpanbecomerowspan/colspan, and cells whosekindiscolumnHeaderorrowHeaderbecome<th>. The table enters the chunk layer as one HTML block and is never cut mid-row. - Figures counted, never faked. Since the result carries no pixels, every entry in
figures[](JSON mode) or every neutralized image reference (markdown mode) is counted as offloaded content incontent_metadata. You always know how many figures your index cannot see. - Hierarchy rebuilt, then chunked. Heading levels are reconciled across page boundaries into one tree — the cross-page link Azure's flat
rolelabels do not provide — and where the analysis under-marks headings, a heading-injection pass restores them. Every sentence is then emitted as a chunk carrying itsdepthandchunk_index, grouped into chunksets: root-to-leaf paths that keep each retrieved passage self-explanatory.
The unit you embed is a chunkset, so no overlap is needed and no fragment arrives without its lineage. On one legal document, the same question was answered with 337 tokens of retrieved context instead of 1,542 from a recursive character splitter, with no information dropped. That is an illustration on one file, not a benchmark; methodology in Document Ingestion & Chunking for RAG.
Because you already paid Azure for the analysis, the OCR front-end is skipped entirely and only the chunking work runs.
Two routing details worth knowing:
- Auto-detection. A JSON upload that structurally fingerprints as an Azure Doc Intelligence result is routed through the external-OCR seam automatically on plain ingest. To opt a look-alike JSON out, set
external_ocr_source: "none". - Explicit endpoint.
chunk_external_ocr_resulttakesexternal_ocr_source: "azure_di". A declared source is shape-validated, so a mismatch 422s instead of guessing. See the API reference.
The Azure AI Document Intelligence RAG pipeline, end to end
The same pipeline with a vector store attached. Qdrant here because POMA ships an integration for it; any store works, and the pipeline recipes below cover thirteen of them.
pip install azure-ai-documentintelligence 'poma[qdrant]'import json
import os
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.ai.documentintelligence.models import DocumentContentFormat
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut
from poma.integrations.qdrant import PomaQdrant
# 1. Analyze.
di = DocumentIntelligenceClient(
endpoint=os.environ["AZURE_DI_ENDPOINT"],
credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
poller = di.begin_analyze_document(
"prebuilt-layout",
body=f,
output_content_format=DocumentContentFormat.MARKDOWN,
)
with open("contract.azure-di.json", "w") as f:
json.dump(poller.result().as_dict(), f)
# 2. Chunk: raw analyzeResult in, chunks and chunksets out.
poma = PrimeCut()
chunked = poma.ingest("contract.azure-di.json")
# 3. Store: hybrid dense + sparse points with hierarchy payloads.
qdrant = PomaQdrant(
url=os.environ["QDRANT_URL"],
api_key=os.environ["QDRANT_API_KEY"],
cloud_inference=False, # False embeds locally via fastembed; works on OSS Qdrant
collection_name="contracts",
dense_model="sentence-transformers/all-MiniLM-L6-v2",
sparse_model="Qdrant/bm25",
dense_size=384,
auto_create_collection=True,
)
qdrant.upsert_poma_points(chunked)
# 4. Retrieve: hits merged along their shared heading path.
cheatsheets = qdrant.get_cheatsheets(
query="What are the early termination conditions?",
limit=3,
)
print(cheatsheets[0]["content"])For a store POMA has no integration for, the loop is the same three lines: embed chunkset.to_embed with your own model and upsert it with file_id, page, depth and chunk_index as metadata. See the Qdrant, LangChain and LlamaIndex integrations, or the per-store recipes below.
Chunking options for Azure Document Intelligence output, compared
| Approach | Structure used | Cross-page hierarchy in the embedded text | Tables | Figures | Page attribution |
|---|---|---|---|---|---|
RecursiveCharacterTextSplitter over content | None | ✗ | Cut mid-row | Silently absent | ✗ |
MarkdownHeaderTextSplitter (Microsoft's recipe) | Header levels | Headers in metadata, not in the vector | Usually survive | Silently absent | Manual |
Hand-rolled sections[] pointer resolver | Full tree, if you build it | Whatever you implement | Yours to reconstruct | Yours to track | Manual |
| PrimeCut on the analyzeResult | Roles, tables, headings, both modes | ✓ chunksets carry the root-to-leaf path | HTML, merged cells intact, never cut | Counted in content_metadata | Kept in the parse (archive chunk records) |
Frequently asked questions
How do I chunk Azure Document Intelligence output for RAG?
Save the raw analyze result and give it to a structure-aware chunker rather than running a character splitter over the content string. PrimeCut auto-detects the Azure shape in either markdown mode or analyzeResult JSON mode, rebuilds the heading hierarchy across pages, keeps tables whole, counts figures as offloaded, and returns chunks plus chunksets that are ready to embed.
What is the recommended Azure Document Intelligence semantic chunking strategy?
Microsoft recommends semantic chunking over fixed-size chunking, using the layout model's markdown output as the substrate and splitting on markdown headers. That is the right starting point. Go one step further and attach the full heading path to the text you actually embed, and cut oversized sections into sentence-level units instead of letting a three-page section become one vector.
How do I do Azure Document Intelligence chunking in LangChain?
Microsoft's sample uses AzureAIDocumentIntelligenceLoader from langchain_community.document_loaders with api_model="prebuilt-layout", then MarkdownHeaderTextSplitter with a headers_to_split_on list. The header metadata it produces lives outside page_content, so concatenate it into the embedded text yourself, or use PomaChunksetSplitter from POMA's LangChain integration, which emits chunksets whose text already contains the heading path.
Should I use markdown mode or analyzeResult JSON for chunking?
Either. Markdown mode is easier to read and is what Microsoft's RAG samples assume; JSON mode gives you role, spans, boundingRegions and tables[] cell grids as data. PrimeCut normalizes both to the same chunks and chunksets: markdown mode is split on PageBreak comments and strips the page-furniture comments, JSON mode is reconstructed from paragraphs[] in span-offset order with roles mapped to headings, furniture paragraphs recorded as page artifacts, and cell grids converted to HTML.
Does the layout model give me a document hierarchy I can chunk on?
Partly. sections[] is a genuine tree, but elements holds JSON pointers such as /paragraphs/0 and /sections/1 rather than text, so you have to resolve them recursively to get a usable path, and the tree does not survive into markdown mode. Paragraph roles are flat labels: a sectionHeading on page 41 has no link to the title on page 3. PrimeCut rebuilds that cross-page tree and puts the resulting path into every chunkset.
What happens to tables and figures during chunking?
Tables survive whole. Markdown-mode HTML tables pass through as-is, and JSON-mode cells[] are rebuilt onto the rowCount × columnCount grid with rowSpan/columnSpan preserved and columnHeader cells rendered as <th>. Figures cannot survive, because the analyze result carries regions and captions rather than pixels — cropped images are a separate output=figures fetch. Every figure is therefore counted as offloaded content in content_metadata, so the gap is visible rather than silent.
Can I chunk Azure Document Intelligence results without re-running the analysis?
Yes, that is the point of the bring-your-own-OCR path. Upload the saved JSON and POMA skips its own OCR front-end, running only the structure, chunking and retrieval-preparation stages. Auto-detection handles it on a plain ingest; to be explicit, call chunk_external_ocr_result with external_ocr_source: "azure_di", which is shape-validated and 422s on a mismatch.
The rest of the series
The optimal chunker, for every OCR and parsing tool you already run:
End-to-end pipeline recipes from Azure Document Intelligence to your vector database:
- Azure Document Intelligence → Chroma
- Azure Document Intelligence → Elasticsearch
- Azure Document Intelligence → LanceDB
- Azure Document Intelligence → Milvus
- Azure Document Intelligence → MongoDB Atlas
- Azure Document Intelligence → OpenSearch
- Azure Document Intelligence → pgvector
- Azure Document Intelligence → Pinecone
- Azure Document Intelligence → Qdrant
- Azure Document Intelligence → Redis
- Azure Document Intelligence → Turbopuffer
- Azure Document Intelligence → Vespa
- Azure Document Intelligence → Weaviate
Or start with the fundamentals: RAG chunking guide · Document ingestion guide · POMA chunksets.