Skip to content

Mistral OCR to Vespa: Chunking for RAG ​

The short answer: Mistral OCR gives you excellent per-page markdown; Vespa gives you one system that natively fuses BM25 and ANN ranking on the same document, at a scale proven for over a decade at Yahoo and Spotify. Wired together naively — flatten, split, feed — the schema you build around the flattened text usually never gets a page field, because Mistral's page index was already gone by the time you designed it. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON, emits hierarchy-preserving chunksets, and records_from_archive() turns them into documents keyed by a deterministic chunkset_uuid — content-free, with the actual text living on a volume assemble() reads back at query time.

What each end of the pipeline actually provides ​

Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, plus inline image bytes when you set include_image_base64. What it doesn't provide: cross-page hierarchy or retrieval units. Details: The Optimal Chunker for Mistral OCR.

Vespa provides a document type per schema with ordinary attribute fields alongside a tensor field carrying the embedding and its own HNSW index, one YQL query that fuses userQuery() BM25 with nearestNeighbor() ANN through a rank profile you write. What it doesn't provide: any opinion about which fields belong in the schema, or where document content should live. Details: Vespa Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — "\n".join(p["markdown"] for p in pages) → fixed-window splitter → a hand-rolled schema with text and embedding fields only → app.feed_iterable(...) — breaks in a pair-specific way:

  • Mistral's page indices vanish at the join, and because the schema is usually designed by looking at the input to feed, not the eventual query needs, teams skip declaring a page attribute field at authoring time — there's nothing in flattened text to prompt for it.
  • Adding it later is a deploy, not a column. Vespa's schema is compiled into an application package; a new attribute field means editing that package, validating it, and redeploying it to the content cluster, then reindexing every existing document to backfill the field.
  • Overlap doubles index pressure in both engines at once. Splitter overlap duplicates spans into both the HNSW graph and the BM25 inverted index simultaneously — Vespa fuses them in one rank profile, so redundant entries pollute both signals of the same fused score, not just one.

The pipeline, end to end ​

bash
pip install mistralai pyvespa
python
import os
from mistralai.client import Mistral  # mistralai>=2 is a namespace package; the client lives in mistralai.client
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from vespa.package import ApplicationPackage, Field, Schema, Document, HNSW, RankProfile
from vespa.deployment import VespaDocker

# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
    model="mistral-ocr-latest",
    document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
    include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
    f.write(ocr.model_dump_json())

# 2. Chunk the raw result and keep the .poma archive — the volume's source of truth.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.mistral-ocr.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")

package = ApplicationPackage(
    name="contracts",
    schema=[Schema(
        name="chunkset",
        document=Document(fields=[
            Field(name="file_id", type="string", indexing=["attribute"]),
            Field(name="chunkset_index", type="int", indexing=["attribute"]),
            Field(
                name="embedding", type="tensor<float>(x[384])",
                indexing=["attribute", "index"],
                ann=HNSW(distance_metric="angular"),
            ),
        ]),
        rank_profiles=[RankProfile(
            name="poma_rank",
            inputs=[("query(q)", "tensor<float>(x[384])")],
            first_phase="closeness(field, embedding)",
        )],
    )],
)
app = VespaDocker().deploy(application_package=package)

# 3. Content-free ingest — the document holds only routing attributes; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
chunkset_docs = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    chunkset_docs.append({"id": r.id, "fields": {
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
        "embedding": embedder.embed([r.text])[0],
    }})
app.feed_iterable(chunkset_docs, schema="chunkset", namespace="contracts")

# 4. Retrieval — YQL must select file_id/chunkset_index explicitly, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
resp = app.query(
    yql="select file_id, chunkset_index from sources * where "
        "{targetHits:100}nearestNeighbor(embedding, q) limit 10",
    ranking="poma_rank",
    body={"input.query(q)": qv},
)
context = assemble(resp, volume=vol)  # -> [{"file_id", "content"}, ...]

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. The document id — chunkset_uuid(file_id, chunkset_index) — is deterministic, so re-feeding the same document never mints duplicate documents.

Metadata mapping: POMA fields → Vespa primitives ​

POMA fieldVespa primitiveWhat it enables
to_embed (via embedder.embed)tensor<float>(x[N]) field with HNSWANN ranking, fusable with bm25() in a rank profile
chunkset_uuid(file_id, chunkset_index)document iddeterministic — same chunkset, same document, across re-feeds
file_idattribute string fieldYQL where scoping to one document
chunkset_indexattribute int fieldmust be named in the YQL select; required by assemble()
chunk content, page/depth lineagenot written to the Vespa document — lives on the Volumefetched by assemble() at retrieval, deduplicated into a cheatsheet

Frequently asked questions ​

How do I get Mistral OCR results into Vespa for RAG? ​

Save the raw /v1/ocr JSON, run PrimeCut().ingest() with download_dir/filename to keep the .poma archive, turn it into records with records_from_archive(), and feed each as a document keyed by chunkset_uuid. Query with an explicit select file_id, chunkset_index and pass the response to assemble().

Why not just split Mistral's markdown and feed it straight into a Vespa schema? ​

Flattening drops the page index before the schema is even designed, so most naive schemas never declare a page field. Adding one later means editing and redeploying the application package, then reindexing to backfill it — Vespa's documented config-propagation tax.

What fields does a Mistral OCR chunkset need in a Vespa schema? ​

file_id and chunkset_index as attribute fields, plus a tensor field with an HNSW index for the embedding. Content itself isn't written into the document under the content-free pattern — it lives on the volume.

Do images in the Mistral OCR result survive the trip to Vespa? ​

Yes — call /v1/ocr with include_image_base64, and POMA folds the figure's description into the chunk's to_embed text, so it becomes part of a normal, rankable document. Images without bytes or annotation become visible, counted markers.

Why does my Vespa assemble() call return empty results after a Mistral OCR ingest? ​

Almost always the YQL select clause: Vespa only returns fields a query explicitly names, so assemble() has nothing to key off unless the query selects file_id and chunkset_index.

Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Weaviate · Mistral OCR → Milvus

Same store, different parser: browse all pipeline recipes — Vespa combo pages for other parsers ship alongside this one.

Foundations: The Optimal Chunker for Mistral OCR · Vespa Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide