Source: http://www.poma-ai.com/docs/pipelines/azure-document-intelligence-to-weaviate

# The Missing Link Between Azure Document Intelligence and Optimal Retrieval in Weaviate

<ByAuthor />

**The short answer:** Azure Document Intelligence's `prebuilt-layout` produces a superb layout analysis; Weaviate brings built-in hybrid search — BM25F keyword scoring fused with vector similarity via the `alpha` blend. Wired together naively — flatten, split, embed — the pipeline sabotages both halves of that hybrid: the vector side embeds context-free fragments, and the BM25F side scores over text polluted by repeated page furniture and orphaned headings. The missing link is POMA: `PrimeCut().ingest()` consumes the raw analyze result (markdown or JSON mode, auto-detected) and emits hierarchy-preserving [chunksets](/learn/chunking/chunksets) that map onto a Weaviate collection with filterable `file_id`/`page`/`depth` properties.

## What each end of the pipeline actually provides

**Azure Document Intelligence** (`prebuilt-layout`) returns two shapes: markdown mode — one reading-order markdown string with inline HTML tables, `<!-- PageBreak -->` delimiters, and `PageHeader`/`PageFooter`/`PageNumber` furniture comments — or classic JSON mode, a `paragraphs[]` array in span-offset order with `title`/`sectionHeading` roles plus `tables[]` cell grids. Multi-column de-interleaving happens server-side. What it doesn't provide: cross-page hierarchy, retrieval units, or figure bytes. Details: [The Optimal Chunker for Azure Document Intelligence](/optimal-chunker-azure-document-intelligence).

**Weaviate** provides collections with typed, filterable properties, named vectors, hybrid BM25F+vector queries with a tunable `alpha`, cross-references between collections, and generative/reranker modules. What it doesn't provide: any opinion about what an object should contain. BM25F and the vector index both operate on whatever text you stored. Details: [The Optimal Chunks for the Best Retrieval in Weaviate](/optimal-chunks-weaviate).

## The naive wiring, and where it breaks

The common recipe — flatten the analyze result to text, run a fixed-size splitter, embed, insert one object per fragment — breaks in a pair-specific way, because Weaviate's hybrid search punishes dirty text twice:

- **Page furniture poisons BM25F.** `prebuilt-layout` explicitly marks running headers, footers, and page numbers — but the naive flatten folds them into body text, so "Contoso Ltd. — Confidential" recurs in nearly every object. BM25F's term statistics are computed over exactly this noise; the keyword half of every hybrid query is skewed by strings that answer nothing.
- **Headings and their bodies land in different objects.** The fixed-size cut separates a `sectionHeading` like "Termination clauses" from the paragraphs beneath it. A keyword query for the clause name BM25F-matches the fragment holding the heading — which contains no answer — while the fragment holding the answer has lost the term entirely. No `alpha` setting can blend its way out of that.
- **Roles that should have become filterable properties are discarded.** Azure hands you `title`/`sectionHeading` roles and per-page anchoring; the flatten throws both away, so the collection ends up with no `depth`, no `page`, and nothing to `Filter` on beyond raw text.
- **Overlap creates near-duplicate objects** that crowd top-k on the vector side and double-count terms on the BM25F side.

Tuning `alpha` on this collection is rearranging deck chairs. The input has to be fixed first.

## The pipeline, end to end

```bash
pip install azure-ai-documentintelligence weaviate-client poma
```

```python
import json
import os

import weaviate
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.query import Filter

# 1. Azure Document Intelligence — your existing call, unchanged.
di = DocumentIntelligenceClient(
    endpoint=os.environ["AZURE_DI_ENDPOINT"],
    credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
    poller = di.begin_analyze_document(
        "prebuilt-layout",
        body=f,
        output_content_format="markdown",  # omit for classic JSON mode — POMA handles both
    )
analysis = poller.result()
with open("contract.azure-di.json", "w") as f:
    json.dump(analysis.as_dict(), f)

# 2. The missing link — raw analyze result in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("contract.azure-di.json")  # Azure shape auto-detected

def embed(text: str) -> list[float]:
    ...  # your embedding model — return a dense vector

# 3. Weaviate — one collection of chunksets, hierarchy as filterable properties.
client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=weaviate.auth.AuthApiKey(os.environ["WEAVIATE_API_KEY"]),
)
chunksets = client.collections.create(
    "ContractChunksets",
    properties=[
        Property(name="content", data_type=DataType.TEXT),
        Property(name="file_id", data_type=DataType.TEXT),
        Property(name="page", data_type=DataType.INT),
        Property(name="depth", data_type=DataType.INT),
        Property(name="chunk_index", data_type=DataType.INT),
    ],
    vectorizer_config=Configure.Vectorizer.none(),  # we bring our own vectors
)
with chunksets.batch.dynamic() as batch:
    for cs in result.chunksets:
        batch.add_object(
            properties={
                "content": cs.to_embed,
                "file_id": cs.file_id,
                "page": cs.page,
                "depth": cs.depth,
                "chunk_index": cs.chunk_index,
            },
            vector=embed(cs.to_embed),
        )

# 4. Hybrid query — BM25F over clean text, fused with vector similarity.
query = "What are the early termination conditions?"
response = chunksets.query.hybrid(
    query=query,
    vector=embed(query),
    alpha=0.5,  # 0 = pure BM25F, 1 = pure vector; tune on your own queries
    limit=5,
    filters=Filter.by_property("file_id").equal(result.chunksets[0].file_id),
)
for obj in response.objects:
    print(obj.properties["page"], obj.properties["content"][:80])
# Merge the retrieved chunksets into one deduplicated cheatsheet before prompting.
```

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately). Markdown mode is split on `PageBreak` comments with furniture comments dropped — which is what keeps BM25F statistics clean; JSON mode is reconstructed from `paragraphs[]` in span-offset order, roles mapped to heading levels, cell grids converted to HTML. Figures — which Azure never returns as bytes — are counted as offloaded in `content_metadata`, visible loss rather than silent. To force or suppress detection, set `external_ocr_source` to `"azure_di"` or `"none"`.

Measured on our reference legal-document benchmark, chunksets plus cheatsheet assembly answer the same query with **337 tokens** of retrieved context instead of **1,542** for a recursive character splitter, with zero information loss — methodology in [Document Ingestion & Chunking for RAG](/document-ingestion-chunking-rag).

## Metadata mapping: POMA fields → Weaviate primitives

| POMA chunk field | Weaviate primitive | What it enables |
| --- | --- | --- |
| `to_embed` | object vector + `content` TEXT property | vector half + BM25F half of hybrid search |
| `file_id` | TEXT property, `Filter.by_property` | scope queries to one document |
| `page` (from `PageBreak` delimiters or span-to-page mapping) | INT property | page-cited answers, page-range filters |
| `depth` (from `title`/`sectionHeading` roles, rebuilt cross-page) | INT property | filter/re-rank by hierarchy level |
| `chunk_index` | INT property | stable ordering at assembly time |
| chunk ↔ chunkset lineage | cross-reference between collections (optional) | walk from retrieval unit to fine-grained chunks |

## Frequently asked questions

### How do I get Azure Document Intelligence results into Weaviate for RAG?

Save the raw analyze result (either mode), run `PrimeCut().ingest()` on it (auto-detected, hierarchy rebuilt), create a collection with `content`/`file_id`/`page`/`depth` properties, batch-insert one object per chunkset with its embedded vector, and query with `query.hybrid`.

### Why does BM25F scoring suffer when Azure Document Intelligence output is fixed-size split?

Flattened page furniture (`PageHeader`/`PageFooter` text) repeats in nearly every object and dominates term statistics, and splitting separates headings from their bodies so keyword matches land on fragments without answers. POMA strips the furniture and keeps every passage attached to its heading path.

### Which Weaviate properties should Azure Document Intelligence chunks map to?

`content` (TEXT, searched by BM25F) plus filterable scalars: `file_id` (TEXT), `page` (INT, recovered from `PageBreak` delimiters or span-to-page mapping), `depth` (INT), `chunk_index` (INT).

### What alpha should a Weaviate hybrid query use for Azure Document Intelligence content?

Start balanced at `alpha=0.5` and tune on your own queries: exact tokens (clause numbers, defined terms) favor BM25F, paraphrase questions favor the vector side. Tuning only means something once chunks are clean — furniture stripped, headings attached, no overlap duplicates.

### Can Weaviate cross-references model POMA chunks and chunksets?

Yes — one chunkset collection for retrieval, optionally a chunk collection cross-referenced to its parent chunkset for fine-grained access. For most pipelines the chunkset collection alone suffices, since every chunkset carries its root-to-leaf lineage.

## Related recipes

Same parser, different store: [Azure Document Intelligence → Qdrant](/pipelines/azure-document-intelligence-to-qdrant) · [Azure Document Intelligence → Pinecone](/pipelines/azure-document-intelligence-to-pinecone) · [Azure Document Intelligence → Chroma](/pipelines/azure-document-intelligence-to-chroma)

Same store, different parser: [Mistral OCR → Weaviate](/pipelines/mistral-ocr-to-weaviate) · [Docling → Weaviate](/pipelines/docling-to-weaviate) · [Marker → Weaviate](/pipelines/marker-to-weaviate)

Foundations: [The Optimal Chunker for Azure Document Intelligence](/optimal-chunker-azure-document-intelligence) · [The Optimal Chunks for Weaviate](/optimal-chunks-weaviate) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)