Source: http://www.poma-ai.com/docs/pipelines/azure-document-intelligence-to-milvus

# The Missing Link Between Azure Document Intelligence and Optimal Retrieval in Milvus

<ByAuthor />

**The short answer:** Azure Document Intelligence gives you an excellent layout analysis; Milvus gives you vector search that scales further than almost anything else. Wired together naively — flatten, split, embed, insert — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy or fills the scalar fields Milvus filters on. The missing link is POMA: `PrimeCut().ingest()` consumes the raw analyze result (auto-detected, markdown or JSON mode), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets), and each one lands as a Milvus entity with `file_id`, `page`, and `depth` scalar fields driving filtered — and optionally partition-routed — search.

## What each end of the pipeline actually provides

**Azure Document Intelligence** (`prebuilt-layout`) returns one of two shapes: markdown mode — a single reading-order markdown string with inline HTML tables, `PageBreak` delimiters, and `PageHeader`/`PageFooter`/`PageNumber` furniture comments — or JSON mode, `paragraphs[]` ordered by span offset with `role` tags plus `tables[]` as cell grids. Multi-column de-interleaving happens server-side. What it doesn't provide: cross-page hierarchy (roles classify paragraphs, they don't nest them), retrieval units, or figure bytes. Details: [The Optimal Chunker for Azure Document Intelligence](/optimal-chunker-azure-document-intelligence).

**Milvus** provides collections with partitions (a partition key hashes entities automatically — natural multi-tenancy), scalar fields whose filter expressions apply *during* the ANN search, a spectrum of index types (HNSW, IVF variants, DiskANN for corpora that outgrow RAM), and — on recent versions — sparse vectors with built-in BM25 for hybrid search. What it doesn't provide: any opinion about what an entity should contain. Details: [The Optimal Chunks for the Best Retrieval in Milvus](/optimal-chunks-milvus).

## The naive wiring, and where it breaks

The common recipe — flatten the analyze result to text, run a fixed-size splitter, embed, insert `(vector, text)` entities — breaks in a pair-specific way:

- **Page furniture becomes entities.** Azure's markdown mode repeats `PageHeader`, `PageFooter`, and `PageNumber` comments on every page. Flattened and split, a 300-page filing embeds its running header hundreds of times — hundreds of near-identical vectors in the HNSW graph, and on the BM25 sparse side of a hybrid search, boilerplate tokens ("Confidential", the company name) repeated per page skew the term statistics of exactly the lane meant to catch rare exact terms.
- **The scalar fields stay empty.** Milvus's distinguishing primitive — boolean filter expressions applied during the ANN search — has nothing to act on when entities carry only a vector and a text blob. No `file_id` scoping, no page citations, no depth filters, and the partition key has no field to hash on.
- **The pipeline is shape-blind.** Teams write a splitter for markdown mode's `content` string, then feed it a JSON-mode result already sitting in blob storage (or vice versa) — and get one giant unsplit string or a crash, depending on the day.

## The pipeline, end to end

```bash
pip install azure-ai-documentintelligence poma pymilvus
```

```python
import json
import os

from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut
from pymilvus import DataType, MilvusClient

# 1. Your existing Azure Document Intelligence call — unchanged.
di = DocumentIntelligenceClient(
    endpoint=os.environ["AZURE_DI_ENDPOINT"],
    credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
    poller = di.begin_analyze_document(
        "prebuilt-layout",
        body=f,
        output_content_format="markdown",  # omit for classic JSON mode — both work
    )
with open("contract.azure-di.json", "w") as f:
    json.dump(poller.result().as_dict(), f)

# 2. The missing link — raw analyze result in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("contract.azure-di.json")  # Azure shape auto-detected

# 3. A chunkset collection: dense vector + hierarchy scalar fields.
milvus = MilvusClient(
    uri=os.environ["MILVUS_URI"],
    token=os.environ["MILVUS_TOKEN"],
)

schema = MilvusClient.create_schema(auto_id=True)
schema.add_field("pk", DataType.INT64, is_primary=True)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=384)
schema.add_field("text", DataType.VARCHAR, max_length=65535)
schema.add_field("file_id", DataType.VARCHAR, max_length=128, is_partition_key=True)
schema.add_field("page", DataType.INT64)
schema.add_field("depth", DataType.INT64)
schema.add_field("chunk_index", DataType.INT64)

index_params = milvus.prepare_index_params()
index_params.add_index(field_name="vector", index_type="HNSW", metric_type="COSINE")

milvus.create_collection(
    collection_name="chunksets",
    schema=schema,
    index_params=index_params,
)

milvus.insert(
    collection_name="chunksets",
    data=[
        {
            "vector": embed(cs.to_embed),  # your embedding model
            "text": cs.to_embed,
            "file_id": cs.file_id,
            "page": cs.page,
            "depth": cs.depth,
            "chunk_index": cs.chunk_index,
        }
        for cs in result.chunksets
    ],
)

# 4. Filtered search — the expression is applied during the ANN search,
#    and the partition key routes it to only the relevant partitions.
hits = milvus.search(
    collection_name="chunksets",
    data=[embed("What does the contract say about early termination?")],
    filter='file_id == "contract.azure-di.json"',
    limit=5,
    output_fields=["text", "file_id", "page", "depth", "chunk_index"],
)
```

On recent Milvus versions, add a `SPARSE_FLOAT_VECTOR` field with a BM25 function to the same schema and issue a hybrid request that fuses dense and sparse rankings — the chunkset design doesn't change, only the query does. And because POMA stripped the page furniture before chunking, the BM25 term statistics reflect real content, not repeated headers.

The last step is client-side assembly: retrieved chunksets share ancestors, so deduplicate the shared lineage and merge the hits into one coherent context block — a **cheatsheet** — before it goes into the prompt. On our reference legal-document benchmark that assembly meant **337 tokens** of context versus **1,542** for a recursive-splitter baseline, with zero information loss — [methodology here](/document-ingestion-chunking-rag).

POMA validates the payload up front (a corrupted or mislabeled upload 422s immediately, never a silently degraded index), keeps Azure's HTML tables whole, and counts Azure's byte-less figures as offloaded content in `content_metadata` — visible loss, never silent. To force or suppress detection, set `external_ocr_source` to `"azure_di"` or `"none"`.

## Metadata mapping: POMA fields → Milvus primitives

| POMA chunk field | Milvus primitive | What it enables |
| --- | --- | --- |
| `to_embed` | `FLOAT_VECTOR` field (+ optional `SPARSE_FLOAT_VECTOR` BM25) | dense + exact-term hybrid retrieval |
| `to_embed` text | `VARCHAR` field (`text`) | hits arrive with their content, no second lookup |
| `file_id` | `VARCHAR` scalar, `is_partition_key=True` | per-document scoping + partition routing |
| `page` (from `PageBreak` / span offsets) | `INT64` scalar, filter expression | page-cited answers, `page <= 20` filters |
| `depth` (from Azure roles + rebuilt tree) | `INT64` scalar, filter expression | `depth <= 2` structural filters, re-ranking |
| `chunk_index` | `INT64` scalar | stable ordering at assembly time |

## Frequently asked questions

### How do I get Azure Document Intelligence results into Milvus for RAG?

Save the raw analyze result JSON, run `PrimeCut().ingest()` on it (auto-detected, hierarchy rebuilt), then embed each chunkset's `to_embed` and insert entities with `file_id`, `page`, `depth`, and `chunk_index` scalar fields. Filtered ANN search plus client-side cheatsheet assembly completes the loop.

### What scalar fields should Azure Document Intelligence chunks carry in Milvus?

`file_id`, `page`, `depth`, `chunk_index`, plus a `VARCHAR` field for the text. Filter expressions like `file_id == "contract-001"` or `depth <= 2` apply during the ANN search itself — POMA emits every one of these fields, with `page` recovered from Azure's `PageBreak` delimiters or span offsets.

### Do Azure Document Intelligence page headers pollute Milvus hybrid BM25 search?

In naive pipelines, yes: markdown mode repeats header/footer/page-number furniture on every page, and once embedded it skews both the HNSW graph and the sparse BM25 term statistics toward boilerplate. POMA drops the furniture comments before chunking, so both lanes index only real content.

### Does markdown mode versus JSON mode matter when loading Azure Document Intelligence into Milvus?

No — POMA auto-detects and handles both, and the resulting chunks and chunksets are identical in shape either way. Your Milvus schema, filter expressions, and retrieval code never change.

### How do I isolate documents or tenants in one Milvus collection for Azure Document Intelligence content?

Declare `file_id` (or a `tenant_id`) as the partition key at collection creation. Milvus hashes entities into partitions automatically and routes filtered queries to only the relevant ones — POMA writes `file_id` on every chunkset, so isolation comes for free.

## Related recipes

Same parser, different store: [Azure Document Intelligence → pgvector](/pipelines/azure-document-intelligence-to-pgvector) · [Azure Document Intelligence → Pinecone](/pipelines/azure-document-intelligence-to-pinecone) · [Azure Document Intelligence → Chroma](/pipelines/azure-document-intelligence-to-chroma)

Same store, different parser: [Mistral OCR → Milvus](/pipelines/mistral-ocr-to-milvus) · [Unstructured.io → Milvus](/pipelines/unstructured-to-milvus) · [Textract → Milvus](/pipelines/textract-to-milvus)

Foundations: [The Optimal Chunker for Azure Document Intelligence](/optimal-chunker-azure-document-intelligence) · [The Optimal Chunks for Milvus](/optimal-chunks-milvus) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)