Source: http://www.poma-ai.com/docs/pipelines/azure-document-intelligence-to-chroma

# The Missing Link Between Azure Document Intelligence and Optimal Retrieval in Chroma

<ByAuthor />

**The short answer:** Azure Document Intelligence gives you an excellent layout analysis; Chroma gives you the fastest path from zero to a working retrieval loop. Wired together naively — flatten, split, `collection.add` — they still produce mediocre RAG, and worse: the corner cut in the prototype notebook is the one that ships. The missing link is POMA: `PrimeCut().ingest()` consumes the raw analyze result (auto-detected, markdown or JSON mode), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets), and those land in a local `PersistentClient` collection with `where`-filterable hierarchy metadata — while the portable `.poma` archive keeps the same chunks ready for whatever store production lands on.

## What each end of the pipeline actually provides

**Azure Document Intelligence** (`prebuilt-layout`) returns one of two shapes: markdown mode — a single reading-order markdown string with inline HTML tables, `PageBreak` delimiters, and page-furniture comments — or JSON mode, `paragraphs[]` ordered by span offset with `role` tags plus `tables[]` as cell grids. Multi-column de-interleaving happens server-side. What it doesn't provide: cross-page hierarchy (roles classify paragraphs, they don't nest them), retrieval units, or figure bytes. Details: [The Optimal Chunker for Azure Document Intelligence](/optimal-chunker-azure-document-intelligence).

**Chroma** provides a local-first, embedded store — `chromadb.PersistentClient(path=...)` needs no server and the same API scales to client/server later — with collections, `where` metadata filters, `where_document` content filters, and default or bring-your-own embedding functions. What it doesn't provide: any opinion about what a document in the collection should contain. Details: [The Optimal Chunks for the Best Retrieval in Chroma](/optimal-chunks-chroma).

## The naive wiring, and where it breaks

The common recipe — take markdown mode's `content` string, run a fixed-size splitter, `collection.add(documents=chunks)` with empty metadata — breaks in a pair-specific way:

- **The page anchor is a comment, and the splitter eats it.** In markdown mode, `<!-- PageBreak -->` comments are the *only* page signal in the entire result. A character splitter treats them as ordinary text and slices through them, so the `metadatas` you pass to Chroma can never include a `page` — and `where` filters, Chroma's main retrieval lever beyond similarity, have nothing to act on. The enterprise question "where in the document does it say that?" becomes unanswerable.
- **Azure's HTML tables are cut mid-row.** `prebuilt-layout`'s best output — intact tables with merged cells — becomes fragments of `<td>` soup, each embedded as a separate near-meaningless document in the collection.
- **The prototype's chunking sticks.** Chroma demos get built in an afternoon and funded by Friday; re-chunking later means re-embedding, re-running every evaluation, and explaining the shift. The splitter in the notebook quietly becomes the production chunker — which is exactly why the prototype is the right place to chunk properly once.

## The pipeline, end to end

```bash
pip install azure-ai-documentintelligence poma chromadb
```

```python
import json
import os

import chromadb
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut

# 1. Your existing Azure Document Intelligence call — unchanged.
di = DocumentIntelligenceClient(
    endpoint=os.environ["AZURE_DI_ENDPOINT"],
    credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
    poller = di.begin_analyze_document(
        "prebuilt-layout",
        body=f,
        output_content_format="markdown",  # omit for classic JSON mode — both work
    )
with open("contract.azure-di.json", "w") as f:
    json.dump(poller.result().as_dict(), f)

# 2. The missing link — raw analyze result in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("contract.azure-di.json")  # Azure shape auto-detected

# 3. A durable local collection — no server, same API as client/server.
chroma = chromadb.PersistentClient(path="./chroma")
collection = chroma.get_or_create_collection("contracts")

# 4. Add chunksets: Chroma embeds `documents` (default embedding function,
#    or bring your own); hierarchy lands as where-filterable metadata.
collection.add(
    documents=[cs.to_embed for cs in result.chunksets],
    metadatas=[
        {"file_id": cs.file_id, "page": cs.page, "depth": cs.depth}
        for cs in result.chunksets
    ],
    ids=[f"{cs.file_id}-{i}" for i, cs in enumerate(result.chunksets)],
)

# 5. Query, scoped by the metadata the naive pipeline never had.
hits = collection.query(
    query_texts=["What does the contract say about early termination?"],
    n_results=5,
    where={"file_id": result.chunksets[0].file_id},
)
```

After retrieval, merge the hits instead of concatenating them: chunksets from the same section share ancestor headings, and deduplicating that shared lineage assembles one coherent context block — a **cheatsheet**. On our reference legal-document benchmark that assembly meant **337 tokens** of context versus **1,542** for a recursive-splitter baseline, with zero information loss — [methodology here](/document-ingestion-chunking-rag).

POMA validates the payload up front (a corrupted or mislabeled upload 422s immediately, never a silently degraded index), handles both Azure shapes, drops the `PageHeader`/`PageFooter`/`PageNumber` furniture, keeps HTML tables whole, and counts Azure's byte-less figures as offloaded content in `content_metadata` — visible loss, never silent. To force or suppress detection, set `external_ocr_source` to `"azure_di"` or `"none"`.

And because every ingest also produces a portable `.poma` archive, the collection you just built is a *projection*, not a commitment: when production lands on Qdrant, pgvector, or Milvus, you re-upsert the same chunksets with the same metadata — no second Azure analysis, no re-chunking, no re-evaluation.

## Metadata mapping: POMA fields → Chroma primitives

| POMA chunk field | Chroma primitive | What it enables |
| --- | --- | --- |
| `to_embed` | `documents=` (embedded by default or custom embedding function) | similarity retrieval without managing vectors yourself |
| `file_id` | scalar metadata, `where={"file_id": ...}` | scope queries to one document |
| `page` (from `PageBreak` / span offsets) | scalar metadata (int), `where={"page": {"$lte": ...}}` | page-cited answers, page-range filters |
| `depth` (from Azure roles + rebuilt tree) | scalar metadata (int) | exclude deep appendix material via `where` |
| `chunk_index` | id construction / scalar metadata | stable ordering at assembly time |
| chunkset text | `where_document={"$contains": ...}` | exact-phrase conditions on content |

## Frequently asked questions

### How do I get Azure Document Intelligence output into Chroma for RAG?

Save the raw analyze result JSON, run `PrimeCut().ingest()` on it (auto-detected, hierarchy rebuilt), then `collection.add` the chunksets to a `PersistentClient` collection — `to_embed` as documents, `file_id`/`page`/`depth` as metadata. Chroma embeds them; `where` filters scope retrieval.

### How do I keep Azure Document Intelligence page numbers filterable in Chroma?

Write them at ingest time. Markdown mode's only page anchors are the `PageBreak` comments, which a fixed-size splitter destroys. POMA splits on those delimiters (or reads span offsets in JSON mode) and emits `page` per chunk, so `where={"page": {"$lte": 20}}` works out of the box.

### Do Azure Document Intelligence tables survive in a Chroma collection?

Yes — POMA keeps markdown mode's inline HTML tables intact, converts JSON mode's cell grids to HTML, and never cuts a table mid-row, so an embedded chunkset contains the whole table. A character splitter slices them into fragments that embed as noise.

### Can I move a Chroma prototype to another vector database without re-running the Azure analysis?

Yes, twice over: the analysis is never re-run (BYOCR works from your saved JSON), and the `.poma` archive makes the chunking portable — moving to Qdrant, pgvector, or Milvus is a re-upsert of the same chunksets, not a re-chunking project.

### Do I need to run a server to test an Azure Document Intelligence RAG pipeline with Chroma?

No — `chromadb.PersistentClient(path=...)` gives you a durable on-disk collection with no server process. One script and one afternoon tell you what proper chunking does for your retrieval; the same API scales to client/server later.

## Related recipes

Same parser, different store: [Azure Document Intelligence → Qdrant](/pipelines/azure-document-intelligence-to-qdrant) · [Azure Document Intelligence → pgvector](/pipelines/azure-document-intelligence-to-pgvector) · [Azure Document Intelligence → Milvus](/pipelines/azure-document-intelligence-to-milvus)

Same store, different parser: [Mistral OCR → Chroma](/pipelines/mistral-ocr-to-chroma) · [LlamaParse → Chroma](/pipelines/llamaparse-to-chroma) · [Marker → Chroma](/pipelines/marker-to-chroma)

Foundations: [The Optimal Chunker for Azure Document Intelligence](/optimal-chunker-azure-document-intelligence) · [The Optimal Chunks for Chroma](/optimal-chunks-chroma) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)