Source: http://www.poma-ai.com/docs/optimal-chunker-azure-document-intelligence

# The Optimal Chunker for Azure Document Intelligence

<ByAuthor />

**The short answer:** the optimal chunker for Azure Document Intelligence is one that consumes the raw analyze result *as-is* — markdown mode or JSON mode — preserves the reading order and tables the layout model recovered, rebuilds the heading hierarchy across pages, and emits retrieval units that carry their own context. POMA's bring-your-own-OCR connector does exactly this: you upload the unmodified `prebuilt-layout` result, and PrimeCut returns hierarchy-preserving **chunks** and **chunksets** — no re-analysis, no content flattening, no hand-rolled splitter.

This page explains what Azure Document Intelligence gives you, where it stops, why the fixed-size splitters in Microsoft's own RAG samples squander it, and how to go from an `begin_analyze_document` call to embedded chunksets in a few lines.

## What Azure Document Intelligence gives you — and where it stops

Azure AI Document Intelligence's `prebuilt-layout` model is the workhorse of enterprise document pipelines on Azure — and one of the strongest table and form extractors available. It returns its analysis in one of **two shapes**, depending on how you call it:

**Markdown mode** (`outputContentFormat="markdown"`): `content` is a single reading-order markdown string for the whole document — headings as `#`/`##`, tables inline as HTML, pages delimited by comment markers:

```markdown
# Annual Report 2025

<!-- PageHeader="Contoso Ltd. — Confidential" -->

## Financial highlights

<table><tr><th>Quarter</th><th>Revenue</th></tr><tr><td>Q1</td><td>…</td></tr></table>

<!-- PageNumber="1" -->
<!-- PageBreak -->
```

**JSON mode** (the default, and what older pipelines have on disk): structure arrives as data, not prose — a `paragraphs[]` array ordered by span offset, each paragraph carrying an optional `role` (`title`, `sectionHeading`, `pageHeader`, `pageFooter`, `pageNumber`), plus `tables[]` as cell grids with row/column indices and spans:

```json
{
  "analyzeResult": {
    "paragraphs": [
      { "role": "title", "content": "Annual Report 2025", "spans": [{ "offset": 0, "length": 18 }] },
      { "role": "sectionHeading", "content": "Financial highlights", "spans": [{ "offset": 20, "length": 20 }] },
      { "content": "Revenue grew across all segments…", "spans": [{ "offset": 42, "length": 33 }] }
    ],
    "tables": [{ "rowCount": 4, "columnCount": 2, "cells": [{ "kind": "columnHeader", "content": "Quarter" }] }]
  }
}
```

Two things Azure does exceptionally well here: **tables** (cell grids with merged-cell spans survive intact) and **multi-column layouts** — de-interleaving happens server-side, so span order in JSON mode and the markdown string in markdown mode both already follow human reading order.

What Azure Document Intelligence deliberately does **not** do:

- **Cross-page hierarchy.** A `sectionHeading` on page 41 has no machine-readable link to the `title` that opened the chapter on page 3. Roles classify paragraphs; they don't nest them.
- **Chunking.** The response is an analysis, not retrieval units. Deciding what an embedding vector should represent is left entirely to you — Microsoft's RAG samples answer that question with fixed-size splitters.
- **Figure content.** The analyze result never contains cropped figure bytes. Figures are regions with bounding boxes and captions; the pixels stay in your source document. Something downstream has to make that loss visible.

The gap between *a great analysis* and *good retrieval* is where enterprise RAG pipelines on Azure quietly fail.

## Why generic text splitters waste prebuilt-layout output

The default move — take `content` (or flatten `paragraphs[]` to text) and run a fixed-size or recursive character splitter, exactly as Microsoft's RAG samples do — destroys most of what the layout model recovered:

| What prebuilt-layout recovered | What a fixed-size splitter does with it |
| --- | --- |
| Paragraph roles (`title`, `sectionHeading`) | Ignored — cuts fall wherever the character count lands |
| HTML tables / cell grids with merged cells | Sliced mid-row; header row separated from data rows |
| `PageBreak` delimiters / span-to-page mapping | Lost at flattening — chunks can't cite a page |
| Server-side multi-column reading order | Preserved only by luck of the cut positions |
| `PageHeader` / `PageFooter` / `PageNumber` furniture | Left in — repeated on every page, embedded as noise |

Markdown-header splitters (splitting on `##` boundaries in markdown mode) are a step up, but they still produce **isolated fragments**: the chunk under `### Early termination` no longer knows it lives inside `## Termination clauses` inside `# Master Services Agreement`. At retrieval time the LLM reads an orphaned paragraph and answers out of context. That failure mode — not extraction accuracy — is where most Azure-based RAG pipelines lose their quality.

For the full taxonomy of these failure modes, see the [RAG chunking strategies guide](/rag-chunking-strategies-text-splitters).

## The optimal chunker: hierarchy-aware chunking on the raw analyze result

POMA's **bring-your-own-OCR** (BYOCR) connector was built for teams who already run their own parsing step — and enterprise Azure shops rarely get to swap out Document Intelligence. The connector consumes the *unmodified* analyze result and runs only POMA's downstream stages on it:

1. **Shape validation, up front.** The payload is fingerprinted (the `analyzeResult` envelope, or `paragraphs[]` carrying `spans`). A corrupted or mislabeled upload fails immediately with a clear 422 — never a silently degraded index.
2. **Both shapes, handled natively.** Markdown mode is split on `<!-- PageBreak -->` delimiters, restoring per-page anchoring, and the `PageHeader`/`PageFooter`/`PageNumber` furniture comments are dropped. JSON mode is reconstructed from `paragraphs[]` in span-offset order — which Azure guarantees is reading order, multi-column included — with `role` mapped to heading levels and `tables[]` cell grids converted to HTML.
3. **Tables survive whole.** Inline HTML tables (markdown mode) and reconstructed cell grids (JSON mode) enter the chunk layer as HTML, merged cells included, so embeddings never see half a table.
4. **Figures counted, never faked.** Because Azure returns no figure bytes, POMA counts every figure as offloaded content in `content_metadata` — from `figures[]` in JSON mode, from neutralized image refs in markdown mode. Content loss is surfaced, never silent.
5. **Noise removal and heading enrichment.** Any running headers or footers that survive as body text are stripped, and where the analysis under-marks headings, POMA's heading-injection pass restores them before chunking — the same treatment every natively ingested document gets.
6. **Hierarchy → chunks + chunksets.** The cross-page heading tree is rebuilt, and every sentence is emitted as a chunk with its `depth`, `page`, and a `to_embed` normalization — grouped into [chunksets](/learn/chunking/chunksets): root-to-leaf paths that keep each retrieved passage self-explanatory.

Because you already ran the analysis, **POMA charges only for the downstream structure and chunking value** — the OCR front-end is skipped entirely.

The result, measured on our reference legal-document benchmark: the same query answered with **337 tokens** of retrieved context instead of **1,542** for a recursive character splitter, with zero information loss. Methodology: [Document Ingestion & Chunking for RAG](/document-ingestion-chunking-rag).

## How to chunk Azure Document Intelligence output with POMA

Run your analysis exactly as you do today, save the raw result, and hand the JSON to PrimeCut:

```python
import json
import os

from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut

# 1. Your existing Azure Document Intelligence call — unchanged.
di = DocumentIntelligenceClient(
    endpoint=os.environ["AZURE_DI_ENDPOINT"],
    credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
    poller = di.begin_analyze_document(
        "prebuilt-layout",
        body=f,
        output_content_format="markdown",  # omit for classic JSON mode — POMA handles both
    )
analysis = poller.result()
with open("contract.azure-di.json", "w") as f:
    json.dump(analysis.as_dict(), f)

# 2. Chunk the raw result — POMA auto-detects the Azure shape.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.azure-di.json")

print(f"chunks: {len(result.chunks)}")
print(f"chunksets: {len(result.chunksets)}")
print(result.chunksets[0].to_embed)  # embedding-ready, hierarchy included
```

Two things worth knowing:

- **Auto-detection.** A JSON upload whose shape structurally fingerprints as an Azure Document Intelligence result (the `analyzeResult` envelope, or `paragraphs[]` carrying `spans`) is routed through the BYOCR seam automatically. To force the declaration — or to opt a look-alike JSON *out* of detection — set the `external_ocr_source` parameter to `"azure_di"` or `"none"` respectively on the ingest request.
- **Dedicated endpoint.** For explicit routing, the API exposes a `chunk_external_ocr_result` endpoint that accepts `external_ocr_source: "azure_di"`. See the [API reference](https://api.poma-ai.com/v3/docs).

From `result.chunks` / `result.chunksets` onward, everything is your stack: your vector store, your embedding model, your retrieval strategy. See the [Qdrant](/sdk/integrations/qdrant), [LangChain](/sdk/integrations/langchain), and [LlamaIndex](/sdk/integrations/llamaindex) integrations.

## Chunking options for Azure Document Intelligence output, compared

| Approach | Structure used | Cross-page hierarchy | Tables | Figures | Page numbers kept |
| --- | --- | --- | --- | --- | --- |
| Fixed-size splitter (Microsoft RAG samples) | None | ✗ | Cut mid-row | Silently absent | ✗ |
| Markdown-header splitter (markdown mode) | Per-section headings | ✗ | Usually survive | Silently absent | Manual |
| Hand-rolled `paragraphs[]` parser | Roles, if you map them | Only what you build | Yours to reconstruct | Yours to track | Manual |
| **POMA BYOCR (PrimeCut)** | **Full heading tree, rebuilt across pages** | **✓ chunksets** | **HTML, merged cells intact, never cut** | **Counted as offloaded, visibly** | **✓ per chunk** |

## Frequently asked questions

### How do I chunk Azure Document Intelligence output for RAG?

Don't run a fixed-size splitter over the `content` string — you'd discard the reading order, headings, and HTML tables `prebuilt-layout` already recovered. Feed the raw analyze result JSON to a structure-aware chunker instead. POMA's BYOCR connector accepts the unmodified response in either mode, rebuilds the heading hierarchy across pages, and emits chunks plus chunksets ready for embedding.

### Should I use markdown mode or JSON mode from prebuilt-layout for RAG chunking?

Either works — POMA auto-detects and handles both. Markdown mode is split on `PageBreak` comments with page-furniture comments dropped; JSON mode is reconstructed from `paragraphs[]` in span-offset order, with `role` mapped to headings and `tables[]` converted to HTML.

### Can I chunk Azure Document Intelligence results without re-running the analysis?

Yes — that's the point of the bring-your-own-OCR path. POMA skips its OCR front-end and runs only the downstream structure, chunking, and retrieval-preparation stages on your saved result. Upload the raw JSON (auto-detected) or declare `external_ocr_source: "azure_di"`.

### How does chunking handle multi-column layouts from Azure Document Intelligence?

Azure de-interleaves multi-column layouts server-side, so span order (JSON mode) and the `content` string (markdown mode) already follow human reading order. POMA relies on that ordering rather than re-guessing it — no interleaved sentences from adjacent columns.

### What happens to figures in an Azure Document Intelligence result during chunking?

The analyze result never contains cropped figure bytes, so POMA counts every figure as offloaded content in `content_metadata` — from `figures[]` in JSON mode, from neutralized refs in markdown mode. Figure loss is always visible, never silent.

### Don't Microsoft's RAG samples already handle chunking?

They handle it with fixed-size or token-window splitters over the markdown output — which discards the structure the layout model just recovered. A hierarchy-preserving chunker keeps every passage linked to its full heading path instead of emitting orphaned fragments.

## The rest of the series

The optimal chunker, for every OCR and parsing tool you already run:

- [Mistral OCR](/optimal-chunker-mistral-ocr)
- [LlamaParse](/optimal-chunker-llamaparse)
- [Unstructured.io](/optimal-chunker-unstructured)
- [Docling](/optimal-chunker-docling)
- [Marker](/optimal-chunker-marker)
- [AWS Textract](/optimal-chunker-textract)

End-to-end pipeline recipes from Azure Document Intelligence to your vector database:

- [Azure Document Intelligence → Qdrant](/pipelines/azure-document-intelligence-to-qdrant)
- [Azure Document Intelligence → Pinecone](/pipelines/azure-document-intelligence-to-pinecone)
- [Azure Document Intelligence → Weaviate](/pipelines/azure-document-intelligence-to-weaviate)
- [Azure Document Intelligence → pgvector](/pipelines/azure-document-intelligence-to-pgvector)
- [Azure Document Intelligence → Milvus](/pipelines/azure-document-intelligence-to-milvus)
- [Azure Document Intelligence → Chroma](/pipelines/azure-document-intelligence-to-chroma)

Or start with the fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [Document ingestion guide](/guides/document-ingestion/) · [POMA chunksets](/learn/chunking/chunksets).