Skip to content

Azure Document Intelligence to Vespa: Chunking for RAG ​

The short answer: Azure Document Intelligence's prebuilt-layout model recovers reading order and tables better than almost any parser on the market; Vespa fuses BM25 and ANN ranking natively, at Yahoo/Spotify scale, in one document. Wired together with a fixed-size splitter, the pair still produces mediocre RAG, because nothing rebuilds the cross-page heading hierarchy Azure discards or keeps Vespa's schema content-free. The missing link is POMA: PrimeCut().ingest() consumes the raw analyze result (auto-detected), emits chunksets into a portable .poma archive, and poma.vektoria feeds content-free documents into Vespa while your document content lives on a volume. Retrieval runs Vespa's own YQL query, then assemble() reconstructs prompt-ready context.

What each end of the pipeline actually provides ​

Azure Document Intelligence returns one of two shapes: markdown mode (outputContentFormat="markdown") — one reading-order string with inline HTML tables and <!-- PageBreak --> delimiters — or JSON mode's paragraphs[], ordered by span offset with role fields for headings. Both de-interleave multi-column layouts server-side. What it never provides: cross-page hierarchy, chunking, or figure bytes. Details: The Optimal Chunker for Azure Document Intelligence.

Vespa provides one document type holding both scalar attributes and tensor fields with HNSW, native BM25 via index: enable-bm25, and multi-phase ranking that fuses both signals inside a single rank profile. What it doesn't provide: any opinion about what a document should contain, or where the actual text lives — vektoria's contract keeps the schema holding only (file_id, chunkset_index, embedding). Details: Vespa Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common shortcut — split Azure's content string on <!-- PageBreak -->, run a fixed-size splitter over each page, embed, feed the result into a text field with enable-bm25 — breaks in a pair-specific way. The PageHeader/PageFooter/PageNumber furniture comments Azure leaves inline survive the split and land inside the same field Vespa's rank profile scores with bm25(text). A first-phase ranking function like closeness(field, embedding) * (1 + bm25(text)) now inflates the score of every document containing the repeated header string, so boilerplate outranks the clause that actually answers the query. Compounding this, Azure never returns cropped figure bytes — a naive pipeline has no image content to feed a tensor field for figures at all, so that content is simply absent, with no signal in the schema that anything was lost.

The pipeline, end to end ​

bash
pip install azure-ai-documentintelligence pyvespa
python
import json
import os

from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from vespa.package import ApplicationPackage, Field, Schema, Document, HNSW, RankProfile
from vespa.deployment import VespaDocker

# 1. Azure Document Intelligence — your existing call, unchanged.
di = DocumentIntelligenceClient(
    endpoint=os.environ["AZURE_DI_ENDPOINT"],
    credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
    poller = di.begin_analyze_document(
        "prebuilt-layout",
        body=f,
        output_content_format="markdown",
    )
analysis = poller.result()
with open("result.azure-di.json", "w") as f:
    json.dump(analysis.as_dict(), f)

# 2. The missing link — raw result JSON in, .poma archive out (auto-detected Azure shape).
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("result.azure-di.json", download_dir="archives", filename="doc.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")

package = ApplicationPackage(
    name="contracts",
    schema=[Schema(
        name="poma",
        document=Document(fields=[
            Field(name="file_id", type="string", indexing=["attribute"]),
            Field(name="chunkset_index", type="int", indexing=["attribute"]),
            Field(
                name="embedding", type="tensor<float>(x[384])",
                indexing=["attribute", "index"],
                ann=HNSW(distance_metric="angular"),
            ),
        ]),
        rank_profiles=[RankProfile(
            name="poma_rank",
            inputs=[("query(q)", "tensor<float>(x[384])")],
            first_phase="closeness(field, embedding)",
        )],
    )],
)
app = VespaDocker().deploy(application_package=package)

# 3. Content-free ingest — Vespa holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/doc.poma")  # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/doc.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    app.feed_data_point(schema="poma", data_id=r.id, fields={
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
        "embedding": embedder.embed([r.text])[0],
    })

# 4. Retrieval — Vespa's normal nearestNeighbor query, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
resp = app.query(yql="select file_id, chunkset_index from poma where "
                      "{targetHits:100}nearestNeighbor(embedding, q) limit 10",
                  ranking="poma_rank", body={"input.query(q)": qv})
context = assemble(resp, volume=vol)  # -> [{"file_id", "content"}, ...]

POMA validates the payload shape up front (a mislabeled upload 422s immediately), splits markdown mode on PageBreak while dropping furniture comments, reconstructs JSON mode from span-ordered paragraphs[], and counts every figure as offloaded content since Azure never returns figure bytes. To force or suppress detection, set external_ocr_source to "azure_di" or "none".

Metadata mapping: POMA fields → Vespa primitives ​

POMA fieldVespa primitiveWhat it enables
id (chunkset_uuid)document data_idstable identity across re-ingests
to_embedtensor<float> field with HNSWnearestNeighbor() ranking
file_idattribute string fieldscope a YQL query to one document
chunkset_indexattribute int field, explicitly selectedassemble() match key
page, depth, chunks, textvolume document, not indexedcontent-free retrieval, cheatsheet assembly

Frequently asked questions ​

How do I get Azure Document Intelligence results into Vespa for RAG? ​

Save the raw analyze result JSON, hand it to PrimeCut().ingest() (auto-detected, hierarchy rebuilt into a .poma archive), then loop records_from_archive() — write content to a volume, feed_data_point() the vector plus file_id/chunkset_index. Retrieve with a nearestNeighbor() YQL query plus assemble().

Why not just feed Azure's markdown content string straight into a Vespa document field? ​

Azure's PageHeader/PageFooter/PageBreak comments survive a naive split and land in the same field a rank profile scores with bm25(text), so repeated boilerplate outranks the real answer. Vespa ranks whatever you feed it.

Should I use streaming or indexed mode for Azure Document Intelligence documents in Vespa? ​

Streaming mode suits many small per-tenant corpora with grouped IDs; indexed mode suits one large shared corpus. This depends on your tenancy shape, not the source parser — don't mix both modes in one content cluster.

What happens to Azure Document Intelligence figures in a Vespa deployment? ​

Nothing reaches Vespa — Azure never returns cropped figure bytes, so there's no image content for a tensor field. POMA counts every figure as offloaded content in content_metadata instead of leaving a silent gap.

Does Vespa retrieval need anything special for POMA's assemble() to work? ​

Yes — select file_id, chunkset_index explicitly in the YQL. Vespa doesn't return arbitrary fields by default, and without the explicit select assemble() gets nothing to match against the volume.

Same parser, different store: Azure Document Intelligence → Weaviate · Azure Document Intelligence → Milvus · Azure Document Intelligence → Turbopuffer

Foundations: The Optimal Chunker for Azure Document Intelligence · Vespa Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide