Mistral OCR to Vespa: Chunking for RAG
The short answer: Mistral OCR gives you excellent per-page markdown; Vespa gives you one system that natively fuses BM25 and ANN ranking on the same document, at a scale proven for over a decade at Yahoo and Spotify. Wired together naively — flatten, split, feed — the schema you build around the flattened text usually never gets a page field, because Mistral's page index was already gone by the time you designed it. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON, emits hierarchy-preserving chunksets, and records_from_archive() turns them into documents keyed by a deterministic chunkset_uuid — content-free, with the actual text living on a volume assemble() reads back at query time.
What each end of the pipeline actually provides
Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, plus inline image bytes when you set include_image_base64. What it doesn't provide: cross-page hierarchy or retrieval units. Details: The Optimal Chunker for Mistral OCR.
Vespa provides a document type per schema with ordinary attribute fields alongside a tensor field carrying the embedding and its own HNSW index, one YQL query that fuses userQuery() BM25 with nearestNeighbor() ANN through a rank profile you write. What it doesn't provide: any opinion about which fields belong in the schema, or where document content should live. Details: Vespa Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — "\n".join(p["markdown"] for p in pages) → fixed-window splitter → a hand-rolled schema with text and embedding fields only → app.feed_iterable(...) — breaks in a pair-specific way:
- Mistral's page indices vanish at the join, and because the schema is usually designed by looking at the input to feed, not the eventual query needs, teams skip declaring a
pageattribute field at authoring time — there's nothing in flattened text to prompt for it. - Adding it later is a deploy, not a column. Vespa's schema is compiled into an application package; a new attribute field means editing that package, validating it, and redeploying it to the content cluster, then reindexing every existing document to backfill the field.
- Overlap doubles index pressure in both engines at once. Splitter overlap duplicates spans into both the HNSW graph and the BM25 inverted index simultaneously — Vespa fuses them in one rank profile, so redundant entries pollute both signals of the same fused score, not just one.
The pipeline, end to end
pip install mistralai pyvespaimport os
from mistralai.client import Mistral # mistralai>=2 is a namespace package; the client lives in mistralai.client
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from vespa.package import ApplicationPackage, Field, Schema, Document, HNSW, RankProfile
from vespa.deployment import VespaDocker
# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
model="mistral-ocr-latest",
document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
f.write(ocr.model_dump_json())
# 2. Chunk the raw result and keep the .poma archive — the volume's source of truth.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.mistral-ocr.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
package = ApplicationPackage(
name="contracts",
schema=[Schema(
name="chunkset",
document=Document(fields=[
Field(name="file_id", type="string", indexing=["attribute"]),
Field(name="chunkset_index", type="int", indexing=["attribute"]),
Field(
name="embedding", type="tensor<float>(x[384])",
indexing=["attribute", "index"],
ann=HNSW(distance_metric="angular"),
),
]),
rank_profiles=[RankProfile(
name="poma_rank",
inputs=[("query(q)", "tensor<float>(x[384])")],
first_phase="closeness(field, embedding)",
)],
)],
)
app = VespaDocker().deploy(application_package=package)
# 3. Content-free ingest — the document holds only routing attributes; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
chunkset_docs = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
chunkset_docs.append({"id": r.id, "fields": {
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
"embedding": embedder.embed([r.text])[0],
}})
app.feed_iterable(chunkset_docs, schema="chunkset", namespace="contracts")
# 4. Retrieval — YQL must select file_id/chunkset_index explicitly, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
resp = app.query(
yql="select file_id, chunkset_index from sources * where "
"{targetHits:100}nearestNeighbor(embedding, q) limit 10",
ranking="poma_rank",
body={"input.query(q)": qv},
)
context = assemble(resp, volume=vol) # -> [{"file_id", "content"}, ...]POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. The document id — chunkset_uuid(file_id, chunkset_index) — is deterministic, so re-feeding the same document never mints duplicate documents.
Metadata mapping: POMA fields → Vespa primitives
| POMA field | Vespa primitive | What it enables |
|---|---|---|
to_embed (via embedder.embed) | tensor<float>(x[N]) field with HNSW | ANN ranking, fusable with bm25() in a rank profile |
chunkset_uuid(file_id, chunkset_index) | document id | deterministic — same chunkset, same document, across re-feeds |
file_id | attribute string field | YQL where scoping to one document |
chunkset_index | attribute int field | must be named in the YQL select; required by assemble() |
| chunk content, page/depth lineage | not written to the Vespa document — lives on the Volume | fetched by assemble() at retrieval, deduplicated into a cheatsheet |
Frequently asked questions
How do I get Mistral OCR results into Vespa for RAG?
Save the raw /v1/ocr JSON, run PrimeCut().ingest() with download_dir/filename to keep the .poma archive, turn it into records with records_from_archive(), and feed each as a document keyed by chunkset_uuid. Query with an explicit select file_id, chunkset_index and pass the response to assemble().
Why not just split Mistral's markdown and feed it straight into a Vespa schema?
Flattening drops the page index before the schema is even designed, so most naive schemas never declare a page field. Adding one later means editing and redeploying the application package, then reindexing to backfill it — Vespa's documented config-propagation tax.
What fields does a Mistral OCR chunkset need in a Vespa schema?
file_id and chunkset_index as attribute fields, plus a tensor field with an HNSW index for the embedding. Content itself isn't written into the document under the content-free pattern — it lives on the volume.
Do images in the Mistral OCR result survive the trip to Vespa?
Yes — call /v1/ocr with include_image_base64, and POMA folds the figure's description into the chunk's to_embed text, so it becomes part of a normal, rankable document. Images without bytes or annotation become visible, counted markers.
Why does my Vespa assemble() call return empty results after a Mistral OCR ingest?
Almost always the YQL select clause: Vespa only returns fields a query explicitly names, so assemble() has nothing to key off unless the query selects file_id and chunkset_index.
Related recipes
Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Weaviate · Mistral OCR → Milvus
Same store, different parser: browse all pipeline recipes — Vespa combo pages for other parsers ship alongside this one.
Foundations: The Optimal Chunker for Mistral OCR · Vespa Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide