Azure Document Intelligence to Weaviate: Chunking for RAG
The short answer: Azure Document Intelligence's prebuilt-layout produces a superb layout analysis; Weaviate brings built-in hybrid search — BM25F keyword scoring fused with vector similarity via the alpha blend. Wired together naively — flatten, split, embed — the pipeline sabotages both halves of that hybrid: the vector side embeds context-free fragments, and the BM25F side scores over text polluted by repeated page furniture and orphaned headings. The missing link is POMA: PrimeCut().ingest() consumes the raw analyze result (markdown or JSON mode, auto-detected) and emits hierarchy-preserving chunksets that map onto a Weaviate collection with filterable file_id/page/depth properties.
What each end of the pipeline actually provides
Azure Document Intelligence (prebuilt-layout) returns two shapes: markdown mode — one reading-order markdown string with inline HTML tables, <!-- PageBreak --> delimiters, and PageHeader/PageFooter/PageNumber furniture comments — or classic JSON mode, a paragraphs[] array in span-offset order with title/sectionHeading roles plus tables[] cell grids. Multi-column de-interleaving happens server-side. What it doesn't provide: cross-page hierarchy, retrieval units, or figure bytes. Details: The Optimal Chunker for Azure Document Intelligence.
Weaviate provides collections with typed, filterable properties, named vectors, hybrid BM25F+vector queries with a tunable alpha, cross-references between collections, and generative/reranker modules. What it doesn't provide: any opinion about what an object should contain. BM25F and the vector index both operate on whatever text you stored. Details: The Optimal Chunks for the Best Retrieval in Weaviate.
The naive wiring, and where it breaks
The common recipe — flatten the analyze result to text, run a fixed-size splitter, embed, insert one object per fragment — breaks in a pair-specific way, because Weaviate's hybrid search punishes dirty text twice:
- Page furniture poisons BM25F.
prebuilt-layoutexplicitly marks running headers, footers, and page numbers — but the naive flatten folds them into body text, so "Contoso Ltd. — Confidential" recurs in nearly every object. BM25F's term statistics are computed over exactly this noise; the keyword half of every hybrid query is skewed by strings that answer nothing. - Headings and their bodies land in different objects. The fixed-size cut separates a
sectionHeadinglike "Termination clauses" from the paragraphs beneath it. A keyword query for the clause name BM25F-matches the fragment holding the heading — which contains no answer — while the fragment holding the answer has lost the term entirely. Noalphasetting can blend its way out of that. - Roles that should have become filterable properties are discarded. Azure hands you
title/sectionHeadingroles and per-page anchoring; the flatten throws both away, so the collection ends up with nodepth, nopage, and nothing toFilteron beyond raw text. - Overlap creates near-duplicate objects that crowd top-k on the vector side and double-count terms on the BM25F side.
Tuning alpha on this collection is rearranging deck chairs. The input has to be fixed first.
The pipeline, end to end
pip install azure-ai-documentintelligence weaviate-client pomaimport json
import os
import weaviate
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.query import Filter
# 1. Azure Document Intelligence — your existing call, unchanged.
di = DocumentIntelligenceClient(
endpoint=os.environ["AZURE_DI_ENDPOINT"],
credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
poller = di.begin_analyze_document(
"prebuilt-layout",
body=f,
output_content_format="markdown", # omit for classic JSON mode — POMA handles both
)
analysis = poller.result()
with open("contract.azure-di.json", "w") as f:
json.dump(analysis.as_dict(), f)
# 2. The missing link — raw analyze result in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest( # Azure shape auto-detected
"contract.azure-di.json",
download_dir="store",
filename="contract.poma", # keep the archive — page numbers live in it
)
# Depth comes off the SDK objects; page is on the archive's chunk records only.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
c["chunk_index"]: c.get("page")
for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}
def embed(text: str) -> list[float]:
... # your embedding model — return a dense vector
# 3. Weaviate — one collection of chunksets, hierarchy as filterable properties.
client = weaviate.connect_to_weaviate_cloud(
cluster_url=os.environ["WEAVIATE_URL"],
auth_credentials=weaviate.auth.AuthApiKey(os.environ["WEAVIATE_API_KEY"]),
)
chunksets = client.collections.create(
"ContractChunksets",
properties=[
Property(name="content", data_type=DataType.TEXT),
Property(name="file_id", data_type=DataType.TEXT),
Property(name="page", data_type=DataType.INT),
Property(name="depth", data_type=DataType.INT),
Property(name="chunkset_index", data_type=DataType.INT),
],
vectorizer_config=Configure.Vectorizer.none(), # we bring our own vectors
)
with chunksets.batch.dynamic() as batch:
for cs in result.chunksets:
leaf = cs.chunks[-1] # document order — the last chunk index is the leaf
batch.add_object(
properties={
"content": cs.to_embed,
"file_id": cs.file_id,
"chunkset_index": cs.chunkset_index,
"page": page_of[leaf],
"depth": depth_of[leaf],
},
vector=embed(cs.to_embed),
)
# 4. Hybrid query — BM25F over clean text, fused with vector similarity.
query = "What are the early termination conditions?"
response = chunksets.query.hybrid(
query=query,
vector=embed(query),
alpha=0.5, # 0 = pure BM25F, 1 = pure vector; tune on your own queries
limit=5,
filters=Filter.by_property("file_id").equal(result.chunksets[0].file_id),
)
for obj in response.objects:
print(obj.properties["page"], obj.properties["content"][:80])
# Merge the retrieved chunksets into one deduplicated cheatsheet before prompting.POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately). Markdown mode is split on PageBreak comments with furniture comments dropped — which is what keeps BM25F statistics clean; JSON mode is reconstructed from paragraphs[] in span-offset order, roles mapped to heading levels, cell grids converted to HTML. Figures — which Azure never returns as bytes — are counted as offloaded in content_metadata, visible loss rather than silent. To force or suppress detection, set external_ocr_source to "azure_di" or "none".
Measured on our reference legal-document benchmark, chunksets plus cheatsheet assembly answer the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss — methodology in Document Ingestion & Chunking for RAG.
Metadata mapping: POMA fields → Weaviate primitives
| POMA chunk field | Weaviate primitive | What it enables |
|---|---|---|
to_embed | object vector + content TEXT property | vector half + BM25F half of hybrid search |
file_id | TEXT property, Filter.by_property | scope queries to one document |
page of the chunkset's leaf chunk (from the archive's chunk records, filled from PageBreak delimiters or span-to-page mapping) | INT property | page-cited answers, page-range filters |
depth of the leaf chunk (from title/sectionHeading roles, rebuilt cross-page) | INT property | filter/re-rank by hierarchy level |
chunkset_index | INT property | stable ordering at assembly time |
| chunk ↔ chunkset lineage | cross-reference between collections (optional) | walk from retrieval unit to fine-grained chunks |
Frequently asked questions
How do I get Azure Document Intelligence results into Weaviate for RAG?
Save the raw analyze result (either mode), run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), create a collection with content/file_id/page/depth properties, batch-insert one object per chunkset with its embedded vector, and query with query.hybrid.
Why does BM25F scoring suffer when Azure Document Intelligence output is fixed-size split?
Flattened page furniture (PageHeader/PageFooter text) repeats in nearly every object and dominates term statistics, and splitting separates headings from their bodies so keyword matches land on fragments without answers. POMA strips the furniture and keeps every passage attached to its heading path.
Which Weaviate properties should Azure Document Intelligence chunks map to?
content (TEXT, searched by BM25F) plus filterable scalars: file_id (TEXT), chunkset_index (INT), page (INT) and depth (INT) taken from the chunkset's leaf chunk — depth off the SDK's chunk objects, page read out of the .poma archive's chunk records, where it was recovered from PageBreak delimiters or span-to-page mapping.
What alpha should a Weaviate hybrid query use for Azure Document Intelligence content?
Start balanced at alpha=0.5 and tune on your own queries: exact tokens (clause numbers, defined terms) favor BM25F, paraphrase questions favor the vector side. Tuning only means something once chunks are clean — furniture stripped, headings attached, no overlap duplicates.
Can Weaviate cross-references model POMA chunks and chunksets?
Yes — one chunkset collection for retrieval, optionally a chunk collection cross-referenced to its parent chunkset for fine-grained access. For most pipelines the chunkset collection alone suffices, since every chunkset carries its root-to-leaf lineage.
Related recipes
Same parser, different store: Azure Document Intelligence → Qdrant · Azure Document Intelligence → Pinecone · Azure Document Intelligence → Chroma
Same store, different parser: Mistral OCR → Weaviate · Docling → Weaviate · Marker → Weaviate
Foundations: The Optimal Chunker for Azure Document Intelligence · The Optimal Chunks for Weaviate · All pipeline recipes · RAG chunking guide