Azure Document Intelligence to Vespa: Chunking for RAG
The short answer: Azure Document Intelligence's prebuilt-layout model recovers reading order and tables better than almost any parser on the market; Vespa fuses BM25 and ANN ranking natively, at Yahoo/Spotify scale, in one document. Wired together with a fixed-size splitter, the pair still produces mediocre RAG, because nothing rebuilds the cross-page heading hierarchy Azure discards or keeps Vespa's schema content-free. The missing link is POMA: PrimeCut().ingest() consumes the raw analyze result (auto-detected), emits chunksets into a portable .poma archive, and poma.vektoria feeds content-free documents into Vespa while your document content lives on a volume. Retrieval runs Vespa's own YQL query, then assemble() reconstructs prompt-ready context.
What each end of the pipeline actually provides
Azure Document Intelligence returns one of two shapes: markdown mode (outputContentFormat="markdown") — one reading-order string with inline HTML tables and <!-- PageBreak --> delimiters — or JSON mode's paragraphs[], ordered by span offset with role fields for headings. Both de-interleave multi-column layouts server-side. What it never provides: cross-page hierarchy, chunking, or figure bytes. Details: The Optimal Chunker for Azure Document Intelligence.
Vespa provides one document type holding both scalar attributes and tensor fields with HNSW, native BM25 via index: enable-bm25, and multi-phase ranking that fuses both signals inside a single rank profile. What it doesn't provide: any opinion about what a document should contain, or where the actual text lives — vektoria's contract keeps the schema holding only (file_id, chunkset_index, embedding). Details: Vespa Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common shortcut — split Azure's content string on <!-- PageBreak -->, run a fixed-size splitter over each page, embed, feed the result into a text field with enable-bm25 — breaks in a pair-specific way. The PageHeader/PageFooter/PageNumber furniture comments Azure leaves inline survive the split and land inside the same field Vespa's rank profile scores with bm25(text). A first-phase ranking function like closeness(field, embedding) * (1 + bm25(text)) now inflates the score of every document containing the repeated header string, so boilerplate outranks the clause that actually answers the query. Compounding this, Azure never returns cropped figure bytes — a naive pipeline has no image content to feed a tensor field for figures at all, so that content is simply absent, with no signal in the schema that anything was lost.
The pipeline, end to end
pip install azure-ai-documentintelligence pyvespaimport json
import os
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from vespa.package import ApplicationPackage, Field, Schema, Document, HNSW, RankProfile
from vespa.deployment import VespaDocker
# 1. Azure Document Intelligence — your existing call, unchanged.
di = DocumentIntelligenceClient(
endpoint=os.environ["AZURE_DI_ENDPOINT"],
credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
poller = di.begin_analyze_document(
"prebuilt-layout",
body=f,
output_content_format="markdown",
)
analysis = poller.result()
with open("result.azure-di.json", "w") as f:
json.dump(analysis.as_dict(), f)
# 2. The missing link — raw result JSON in, .poma archive out (auto-detected Azure shape).
client = PrimeCut() # reads POMA_API_KEY
client.ingest("result.azure-di.json", download_dir="archives", filename="doc.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
package = ApplicationPackage(
name="contracts",
schema=[Schema(
name="poma",
document=Document(fields=[
Field(name="file_id", type="string", indexing=["attribute"]),
Field(name="chunkset_index", type="int", indexing=["attribute"]),
Field(
name="embedding", type="tensor<float>(x[384])",
indexing=["attribute", "index"],
ann=HNSW(distance_metric="angular"),
),
]),
rank_profiles=[RankProfile(
name="poma_rank",
inputs=[("query(q)", "tensor<float>(x[384])")],
first_phase="closeness(field, embedding)",
)],
)],
)
app = VespaDocker().deploy(application_package=package)
# 3. Content-free ingest — Vespa holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/doc.poma") # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/doc.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
app.feed_data_point(schema="poma", data_id=r.id, fields={
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
"embedding": embedder.embed([r.text])[0],
})
# 4. Retrieval — Vespa's normal nearestNeighbor query, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
resp = app.query(yql="select file_id, chunkset_index from poma where "
"{targetHits:100}nearestNeighbor(embedding, q) limit 10",
ranking="poma_rank", body={"input.query(q)": qv})
context = assemble(resp, volume=vol) # -> [{"file_id", "content"}, ...]POMA validates the payload shape up front (a mislabeled upload 422s immediately), splits markdown mode on PageBreak while dropping furniture comments, reconstructs JSON mode from span-ordered paragraphs[], and counts every figure as offloaded content since Azure never returns figure bytes. To force or suppress detection, set external_ocr_source to "azure_di" or "none".
Metadata mapping: POMA fields → Vespa primitives
| POMA field | Vespa primitive | What it enables |
|---|---|---|
id (chunkset_uuid) | document data_id | stable identity across re-ingests |
to_embed | tensor<float> field with HNSW | nearestNeighbor() ranking |
file_id | attribute string field | scope a YQL query to one document |
chunkset_index | attribute int field, explicitly selected | assemble() match key |
page, depth, chunks, text | volume document, not indexed | content-free retrieval, cheatsheet assembly |
Frequently asked questions
How do I get Azure Document Intelligence results into Vespa for RAG?
Save the raw analyze result JSON, hand it to PrimeCut().ingest() (auto-detected, hierarchy rebuilt into a .poma archive), then loop records_from_archive() — write content to a volume, feed_data_point() the vector plus file_id/chunkset_index. Retrieve with a nearestNeighbor() YQL query plus assemble().
Why not just feed Azure's markdown content string straight into a Vespa document field?
Azure's PageHeader/PageFooter/PageBreak comments survive a naive split and land in the same field a rank profile scores with bm25(text), so repeated boilerplate outranks the real answer. Vespa ranks whatever you feed it.
Should I use streaming or indexed mode for Azure Document Intelligence documents in Vespa?
Streaming mode suits many small per-tenant corpora with grouped IDs; indexed mode suits one large shared corpus. This depends on your tenancy shape, not the source parser — don't mix both modes in one content cluster.
What happens to Azure Document Intelligence figures in a Vespa deployment?
Nothing reaches Vespa — Azure never returns cropped figure bytes, so there's no image content for a tensor field. POMA counts every figure as offloaded content in content_metadata instead of leaving a silent gap.
Does Vespa retrieval need anything special for POMA's assemble() to work?
Yes — select file_id, chunkset_index explicitly in the YQL. Vespa doesn't return arbitrary fields by default, and without the explicit select assemble() gets nothing to match against the volume.
Related recipes
Same parser, different store: Azure Document Intelligence → Weaviate · Azure Document Intelligence → Milvus · Azure Document Intelligence → Turbopuffer
Foundations: The Optimal Chunker for Azure Document Intelligence · Vespa Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide