Source: http://www.poma-ai.com/docs/optimal-chunker-azure-document-intelligence # Azure Document Intelligence Chunking: Layout Output to RAG Azure Document Intelligence chunking is the step Azure leaves to you. The `prebuilt-layout` model returns an excellent analysis — reading order, paragraph roles, HTML tables, a section tree — but an analysis is not a set of retrieval units. Something downstream has to decide what one embedding vector represents, and that decision, not extraction accuracy, is where most Azure Document Intelligence RAG pipelines lose quality. This page covers three things: the quickest path from a `prebuilt-layout` result to embeddable chunks, Microsoft's own recommended semantic chunking recipe and where its ceiling is, and a full pipeline from `begin_analyze_document` to retrieval. ## Quick start: prebuilt-layout output to chunks Run the analysis exactly as you do today, save the raw result, hand it to PrimeCut. The Azure shape is auto-detected, so there is no adapter to write. ```python import json import os from azure.ai.documentintelligence import DocumentIntelligenceClient from azure.ai.documentintelligence.models import DocumentContentFormat from azure.core.credentials import AzureKeyCredential from poma import PrimeCut # 1. Your existing Azure Document Intelligence call, unchanged. di = DocumentIntelligenceClient( endpoint=os.environ["AZURE_DI_ENDPOINT"], credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]), ) with open("contract.pdf", "rb") as f: poller = di.begin_analyze_document( "prebuilt-layout", body=f, output_content_format=DocumentContentFormat.MARKDOWN, # omit for classic JSON mode ) result = poller.result() with open("contract.azure-di.json", "w") as f: json.dump(result.as_dict(), f) # 2. Chunk the saved analyzeResult. Azure's OCR is not re-run. poma = PrimeCut() # reads POMA_API_KEY chunked = poma.ingest("contract.azure-di.json") print(len(chunked.chunks), len(chunked.chunksets)) print(chunked.chunksets[0].to_embed) # embedding-ready text, heading path included ``` Both output modes work. `output_content_format=DocumentContentFormat.MARKDOWN` gives one reading-order markdown string; omitting it gives the classic `analyzeResult` JSON with `paragraphs[]` and `tables[]`. PrimeCut detects which one it received and normalizes both to the same chunks and chunksets, so a pipeline that switches modes later does not change its index shape. ## What the layout model returns, and where it stops The [layout model](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept-layout) of Azure AI Document Intelligence is one of the strongest table and structure extractors available, and it is worth being precise about what lands in the response. **Markdown mode.** `outputContentFormat=markdown` puts the whole document into `content` as a single reading-order markdown string. Since v4.0 `2024-11-30` (GA), tables are rendered as HTML tables so merged cells and multirow headers survive, and selection marks use the Unicode characters ☒ and ☐. Page furniture arrives as HTML comments: ```markdown # Annual Report 2025 ## Financial highlights
QuarterRevenue
Q14.1M
``` **JSON mode.** The default response is structure as data, not prose: - `paragraphs[]` — each with `content`, `spans` (`offset`, `length` into the top-level content) and `boundingRegions` (carrying `pageNumber`). An optional `role` is one of `title`, `sectionHeading`, `footnote`, `pageHeader`, `pageFooter`, `pageNumber`, `formulaBlock`. - `tables[]` — `rowCount`, `columnCount`, and `cells[]` with `rowIndex`, `columnIndex`, `rowSpan`, `columnSpan` and a `kind` (`columnHeader`, `rowHeader`, `stubHead`, `description`, `content`). - `sections[]` — a hierarchy where "the hierarchical structure is maintained in `elements` for each section". The entries are JSON pointers: `"/paragraphs/0"`, `"/sections/1"`, `"/sections/2"`. - `figures[]` — `id`, `boundingRegions`, `spans`, `elements` and an optional `caption`. - `pages[]` — `pageNumber`, plus `lines`, `words` and `selectionMarks`. Two things Azure does exceptionally well: tables, and multi-column reading order. De-interleaving happens server-side, so span offsets in JSON mode and the `content` string in markdown mode already follow the order a human reads. Three things it deliberately does not do: - **Chunking.** The response is an analysis. Nothing in it says where a retrieval unit begins or ends. - **Resolved hierarchy on the text.** `sections[]` really is a tree, but it is a tree of pointers next to the text, not attached to it. A `sectionHeading` paragraph carries no field naming the chapter it belongs to. To use the tree you resolve every `/paragraphs/N` pointer yourself, recursively, and rebuild the path — and none of it survives into markdown mode, where the tree collapses into `#` levels. - **Figure pixels in the result.** Figures are regions with captions. Cropped images exist only when you pass `output=figures`, and then they are a separate fetch from `/analyzeResults/{resultId}/figures/{figureId}` — never bytes inside the JSON you saved. ## Microsoft's recommended semantic chunking, and its ceiling Microsoft's [RAG guidance for Document Intelligence](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept-retrieval-augmented-generation) is explicit and, as far as it goes, correct: prefer semantic chunking over fixed-size chunking, and use the layout model's markdown output as the substrate. Fixed-size windows, the docs note, "can result in severing words, sentences, or paragraphs". The recommended recipe splits that markdown on its headers, using the LangChain document loader: ```python from langchain_community.document_loaders import AzureAIDocumentIntelligenceLoader from langchain.text_splitter import MarkdownHeaderTextSplitter loader = AzureAIDocumentIntelligenceLoader( file_path="contract.pdf", api_key=key, api_endpoint=endpoint, api_model="prebuilt-layout", ) docs = loader.load() headers_to_split_on = [("#", "Header 1"), ("##", "Header 2"), ("###", "Header 3")] text_splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers_to_split_on) splits = text_splitter.split_text(docs[0].page_content) ``` That `from langchain.text_splitter import ...` line is Microsoft's sample as published; on current LangChain the splitters moved out of the `langchain` package, so import from `langchain_text_splitters` instead. This is a real improvement over a character splitter, and if you are starting from zero it is the right first move. Its ceiling shows up at retrieval time. `MarkdownHeaderTextSplitter` gives each split a `metadata` dict with the headers above it, which sounds like hierarchy. Two problems follow. First, the metadata is not in `page_content`, so unless you concatenate it by hand the embedded text is still an orphan fragment: the paragraph under `### Early termination` is vectorized without the words "Master Services Agreement" or "Termination clauses" anywhere in it, and it will not match a query phrased in those terms. Second, a section is not a chunk. A three-page `## Indemnification` section becomes one enormous split that dilutes into an unusable vector; a two-line `### Notices` becomes a tiny one. You end up bolting a `RecursiveCharacterTextSplitter` onto the output to cut the long ones, and that second pass re-introduces exactly the mid-table, mid-sentence cuts semantic chunking was meant to avoid. Everything else in the analyzeResult is gone by then too. Page numbers went with the `` comments, so no answer can cite "page 41". `PageHeader` and `PageFooter` furniture stayed in the text and gets embedded on every page as noise. The `sections[]` tree was never consulted. For the general taxonomy of these failure modes, see the [RAG chunking strategies guide](/rag-chunking-strategies-text-splitters). ## Azure Document Intelligence chunking on the raw analyzeResult POMA's bring-your-own-OCR path exists for teams who already run a parser and are not going to swap it — and enterprise Azure shops rarely get to swap out Document Intelligence. You upload the unmodified result and only POMA's downstream stages run on it. Concretely, for `azure_di`: 1. **Shape validation up front.** The payload is fingerprinted on the `analyzeResult` envelope, or on `paragraphs[]` carrying Azure's `spans` shape. A mislabeled or corrupted upload fails with a 422, never a quietly degraded index. 2. **Markdown mode.** Detected on `contentFormat: "markdown"`. `content` is split on ``, which restores real page anchoring, and the `PageHeader`, `PageFooter` and `PageNumber` comments are removed rather than embedded. 3. **JSON mode.** Paragraphs are ordered by `spans[0].offset` — reading order, multi-column included, straight from Azure, no geometry re-sorting. `role: "title"` maps to `#` and `role: "sectionHeading"` to `##`. `pageHeader`, `pageFooter` and `pageNumber` paragraphs are routed to metadata instead of the content flow. Each element's page comes from `boundingRegions[0].pageNumber`. 4. **Tables reconstructed whole.** `cells[]` are laid back onto a `rowCount` × `columnCount` grid, `rowSpan` and `columnSpan` become `rowspan`/`colspan`, and cells whose `kind` is `columnHeader` or `rowHeader` become ``. The table enters the chunk layer as one HTML block and is never cut mid-row. 5. **Figures counted, never faked.** Since the result carries no pixels, every entry in `figures[]` (JSON mode) or every neutralized image reference (markdown mode) is counted as offloaded content in `content_metadata`. You always know how many figures your index cannot see. 6. **Hierarchy rebuilt, then chunked.** Heading levels are reconciled across page boundaries into one tree — the cross-page link Azure's flat `role` labels do not provide — and where the analysis under-marks headings, a heading-injection pass restores them. Every sentence is then emitted as a chunk carrying its `depth` and `chunk_index`, grouped into [chunksets](/learn/chunking/chunksets): root-to-leaf paths that keep each retrieved passage self-explanatory. The unit you embed is a chunkset, so no overlap is needed and no fragment arrives without its lineage. On one legal document, the same question was answered with **337 tokens** of retrieved context instead of **1,542** from a recursive character splitter, with no information dropped. That is an illustration on one file, not a benchmark; methodology in [Document Ingestion & Chunking for RAG](/document-ingestion-chunking-rag). Because you already paid Azure for the analysis, the OCR front-end is skipped entirely and only the chunking work runs. Two routing details worth knowing: - **Auto-detection.** A JSON upload that structurally fingerprints as an Azure Doc Intelligence result is routed through the external-OCR seam automatically on plain ingest. To opt a look-alike JSON out, set `external_ocr_source: "none"`. - **Explicit endpoint.** `chunk_external_ocr_result` takes `external_ocr_source: "azure_di"`. A declared source is shape-validated, so a mismatch 422s instead of guessing. See the [API reference](https://api.poma-ai.com/v3/docs). ## The Azure AI Document Intelligence RAG pipeline, end to end The same pipeline with a vector store attached. Qdrant here because POMA ships an integration for it; any store works, and the [pipeline recipes](#the-rest-of-the-series) below cover thirteen of them. ```bash pip install azure-ai-documentintelligence 'poma[qdrant]' ``` ```python import json import os from azure.ai.documentintelligence import DocumentIntelligenceClient from azure.ai.documentintelligence.models import DocumentContentFormat from azure.core.credentials import AzureKeyCredential from poma import PrimeCut from poma.integrations.qdrant import PomaQdrant # 1. Analyze. di = DocumentIntelligenceClient( endpoint=os.environ["AZURE_DI_ENDPOINT"], credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]), ) with open("contract.pdf", "rb") as f: poller = di.begin_analyze_document( "prebuilt-layout", body=f, output_content_format=DocumentContentFormat.MARKDOWN, ) with open("contract.azure-di.json", "w") as f: json.dump(poller.result().as_dict(), f) # 2. Chunk: raw analyzeResult in, chunks and chunksets out. poma = PrimeCut() chunked = poma.ingest("contract.azure-di.json") # 3. Store: hybrid dense + sparse points with hierarchy payloads. qdrant = PomaQdrant( url=os.environ["QDRANT_URL"], api_key=os.environ["QDRANT_API_KEY"], cloud_inference=False, # False embeds locally via fastembed; works on OSS Qdrant collection_name="contracts", dense_model="sentence-transformers/all-MiniLM-L6-v2", sparse_model="Qdrant/bm25", dense_size=384, auto_create_collection=True, ) qdrant.upsert_poma_points(chunked) # 4. Retrieve: hits merged along their shared heading path. cheatsheets = qdrant.get_cheatsheets( query="What are the early termination conditions?", limit=3, ) print(cheatsheets[0]["content"]) ``` For a store POMA has no integration for, the loop is the same three lines: embed `chunkset.to_embed` with your own model and upsert it with `file_id`, `page`, `depth` and `chunk_index` as metadata. See the [Qdrant](/sdk/integrations/qdrant), [LangChain](/sdk/integrations/langchain) and [LlamaIndex](/sdk/integrations/llamaindex) integrations, or the per-store recipes below. ## Chunking options for Azure Document Intelligence output, compared | Approach | Structure used | Cross-page hierarchy in the embedded text | Tables | Figures | Page attribution | | --- | --- | --- | --- | --- | --- | | `RecursiveCharacterTextSplitter` over `content` | None | ✗ | Cut mid-row | Silently absent | ✗ | | `MarkdownHeaderTextSplitter` (Microsoft's recipe) | Header levels | Headers in `metadata`, not in the vector | Usually survive | Silently absent | Manual | | Hand-rolled `sections[]` pointer resolver | Full tree, if you build it | Whatever you implement | Yours to reconstruct | Yours to track | Manual | | **PrimeCut on the analyzeResult** | **Roles, tables, headings, both modes** | **✓ chunksets carry the root-to-leaf path** | **HTML, merged cells intact, never cut** | **Counted in `content_metadata`** | **Kept in the parse (archive chunk records)** | ## Frequently asked questions ### How do I chunk Azure Document Intelligence output for RAG? Save the raw analyze result and give it to a structure-aware chunker rather than running a character splitter over the `content` string. PrimeCut auto-detects the Azure shape in either markdown mode or `analyzeResult` JSON mode, rebuilds the heading hierarchy across pages, keeps tables whole, counts figures as offloaded, and returns chunks plus chunksets that are ready to embed. ### What is the recommended Azure Document Intelligence semantic chunking strategy? Microsoft recommends semantic chunking over fixed-size chunking, using the layout model's markdown output as the substrate and splitting on markdown headers. That is the right starting point. Go one step further and attach the full heading path to the text you actually embed, and cut oversized sections into sentence-level units instead of letting a three-page section become one vector. ### How do I do Azure Document Intelligence chunking in LangChain? Microsoft's sample uses `AzureAIDocumentIntelligenceLoader` from `langchain_community.document_loaders` with `api_model="prebuilt-layout"`, then `MarkdownHeaderTextSplitter` with a `headers_to_split_on` list. The header metadata it produces lives outside `page_content`, so concatenate it into the embedded text yourself, or use `PomaChunksetSplitter` from POMA's LangChain integration, which emits chunksets whose text already contains the heading path. ### Should I use markdown mode or analyzeResult JSON for chunking? Either. Markdown mode is easier to read and is what Microsoft's RAG samples assume; JSON mode gives you `role`, `spans`, `boundingRegions` and `tables[]` cell grids as data. PrimeCut normalizes both to the same chunks and chunksets: markdown mode is split on `PageBreak` comments and strips the page-furniture comments, JSON mode is reconstructed from `paragraphs[]` in span-offset order with roles mapped to headings, furniture paragraphs recorded as page artifacts, and cell grids converted to HTML. ### Does the layout model give me a document hierarchy I can chunk on? Partly. `sections[]` is a genuine tree, but `elements` holds JSON pointers such as `/paragraphs/0` and `/sections/1` rather than text, so you have to resolve them recursively to get a usable path, and the tree does not survive into markdown mode. Paragraph roles are flat labels: a `sectionHeading` on page 41 has no link to the `title` on page 3. PrimeCut rebuilds that cross-page tree and puts the resulting path into every chunkset. ### What happens to tables and figures during chunking? Tables survive whole. Markdown-mode HTML tables pass through as-is, and JSON-mode `cells[]` are rebuilt onto the `rowCount` × `columnCount` grid with `rowSpan`/`columnSpan` preserved and `columnHeader` cells rendered as ``. Figures cannot survive, because the analyze result carries regions and captions rather than pixels — cropped images are a separate `output=figures` fetch. Every figure is therefore counted as offloaded content in `content_metadata`, so the gap is visible rather than silent. ### Can I chunk Azure Document Intelligence results without re-running the analysis? Yes, that is the point of the bring-your-own-OCR path. Upload the saved JSON and POMA skips its own OCR front-end, running only the structure, chunking and retrieval-preparation stages. Auto-detection handles it on a plain ingest; to be explicit, call `chunk_external_ocr_result` with `external_ocr_source: "azure_di"`, which is shape-validated and 422s on a mismatch. ## The rest of the series The optimal chunker, for every OCR and parsing tool you already run: - [AWS Textract](/optimal-chunker-textract) - [DeepSeek-OCR](/optimal-chunker-deepseek-ocr) - [Docling](/optimal-chunker-docling) - [LlamaParse](/optimal-chunker-llamaparse) - [Marker](/optimal-chunker-marker) - [MinerU](/optimal-chunker-mineru) - [Mistral OCR](/optimal-chunker-mistral-ocr) - [PaddleOCR-VL](/optimal-chunker-paddleocr-vl) - [Unstructured.io](/optimal-chunker-unstructured) End-to-end pipeline recipes from Azure Document Intelligence to your vector database: - [Azure Document Intelligence → Chroma](/pipelines/azure-document-intelligence-to-chroma) - [Azure Document Intelligence → Elasticsearch](/pipelines/azure-document-intelligence-to-elasticsearch) - [Azure Document Intelligence → LanceDB](/pipelines/azure-document-intelligence-to-lancedb) - [Azure Document Intelligence → Milvus](/pipelines/azure-document-intelligence-to-milvus) - [Azure Document Intelligence → MongoDB Atlas](/pipelines/azure-document-intelligence-to-mongodb-atlas) - [Azure Document Intelligence → OpenSearch](/pipelines/azure-document-intelligence-to-opensearch) - [Azure Document Intelligence → pgvector](/pipelines/azure-document-intelligence-to-pgvector) - [Azure Document Intelligence → Pinecone](/pipelines/azure-document-intelligence-to-pinecone) - [Azure Document Intelligence → Qdrant](/pipelines/azure-document-intelligence-to-qdrant) - [Azure Document Intelligence → Redis](/pipelines/azure-document-intelligence-to-redis) - [Azure Document Intelligence → Turbopuffer](/pipelines/azure-document-intelligence-to-turbopuffer) - [Azure Document Intelligence → Vespa](/pipelines/azure-document-intelligence-to-vespa) - [Azure Document Intelligence → Weaviate](/pipelines/azure-document-intelligence-to-weaviate) Or start with the fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [Document ingestion guide](/guides/document-ingestion/) · [POMA chunksets](/learn/chunking/chunksets).