Mistral OCR to Turbopuffer: Chunking for RAG
The short answer: Mistral OCR gives you excellent per-page markdown; Turbopuffer gives you a namespace-per-tenant vector store with native hybrid ANN+BM25 ranking at object-storage scale. Wired together naively — flatten, split, embed — you lose Mistral's page indices and end up with unstable row ids that duplicate on every re-ingest. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON, emits hierarchy-preserving chunksets, and records_from_archive() turns them into rows keyed by a deterministic chunkset_uuid — content-free, with the actual text living on a volume assemble() reads back at query time.
What each end of the pipeline actually provides
Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, plus inline image bytes when you set include_image_base64, and a side-channel tables[] spliced at ref. What it doesn't provide: cross-page hierarchy or retrieval units. Details: The Optimal Chunker for Mistral OCR.
Turbopuffer provides namespaces (the tenant/corpus boundary, created implicitly on write), typed attributes declared inline at write time, and native hybrid search — rank_by=("vector","ANN",…), SparseKNN, and BM25 fused server-side via rerank_by=("RRF",). What it doesn't provide: any opinion about what a row's id or attributes should be, or where the actual document content should live. Details: Turbopuffer Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — "\n".join(p["markdown"] for p in pages) → fixed-window splitter → ns.write(upsert_rows=[{"id": hash(chunk), ...}]) — breaks in a pair-specific way:
- Mistral's page indices vanish at the join, so there's no natural key to derive a stable row id from — pipelines fall back to a content hash or a positional counter.
- Ids drift on re-ingest. Any reflow of the source markdown (a re-run, a Mistral model update) shifts window boundaries, minting new ids for content that's substantively unchanged. Turbopuffer's SPFresh index is incrementally updatable and never does a full rebuild, so the old rows aren't reconciled away — they sit in the namespace as dead weight.
- Large inline tables blow the filterable-value cap. Mistral splices
tables[].contentinline; a fixed-size window straddling a wide table often exceeds Turbopuffer's 4 KiB cap on filterable attribute values if that raw window text is (wrongly) marked filterable, and the write is rejected for exactly the documents with the densest tables.
The pipeline, end to end
pip install mistralai turbopufferimport os
from mistralai import Mistral
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
import turbopuffer
# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
model="mistral-ocr-latest",
document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
f.write(ocr.model_dump_json())
# 2. Chunk the raw result and keep the .poma archive — the volume's source of truth.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.mistral-ocr.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
tpuf = turbopuffer.Turbopuffer(api_key=os.environ["TURBOPUFFER_API_KEY"], region="gcp-us-central1")
ns = tpuf.namespace("contracts")
# 3. Content-free ingest — namespace holds only routing attributes; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
rows = []
for r in records:
vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
"chunks": r.payload["chunks"], "text": r.text})
rows.append({
"id": r.id,
"vector": embedder.embed([r.text])[0],
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
})
ns.write(
upsert_rows=rows,
distance_metric="cosine_distance",
schema={"file_id": {"type": "string"}, "chunkset_index": {"type": "int"}},
)
# 4. Retrieval — ANN query with attributes requested, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
res = ns.query(rank_by=("vector", "ANN", qv), top_k=10,
include_attributes=["file_id", "chunkset_index"])
context = assemble(res, volume=vol) # -> [{"file_id", "content"}, ...]POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. The row id — chunkset_uuid(file_id, chunkset_index) — is deterministic, so re-ingesting the same document never mints duplicate rows.
Metadata mapping: POMA fields → Turbopuffer primitives
| POMA field | Turbopuffer primitive | What it enables |
|---|---|---|
to_embed (via embedder.embed) | vector | dense ANN ranking, fusable with BM25/SparseKNN |
chunkset_uuid(file_id, chunkset_index) | row id | deterministic — same chunkset, same row, across re-ingests |
file_id | attribute (string, filterable) | Eq/In scoping to one document |
chunkset_index | attribute (int, filterable) | requested via include_attributes; required by assemble() |
| chunk content, page/depth lineage | not written to Turbopuffer — lives on the Volume | fetched by assemble() at retrieval, deduplicated into a cheatsheet |
Frequently asked questions
How do I get Mistral OCR results into Turbopuffer for RAG?
Save the raw /v1/ocr JSON, run PrimeCut().ingest() with download_dir/filename to keep the .poma archive, turn it into records with records_from_archive(), and write each as a row keyed by chunkset_uuid. Query with include_attributes=["file_id","chunkset_index"] and pass the response to assemble().
Why not just split Mistral's markdown and write it straight into a Turbopuffer namespace?
Flattening destroys the natural key a stable row id would come from. Naive pipelines hash the flattened window instead, so re-ingests mint new ids for shifted boundaries, and Turbopuffer's incrementally-updated SPFresh index never reconciles the resulting duplicates away.
What Turbopuffer attributes should Mistral OCR chunksets carry?
file_id and chunkset_index as compact filterable attributes, both well under the 4 KiB filterable-value cap. Content itself isn't written to Turbopuffer under the content-free pattern — it lives on the volume.
Do images in the Mistral OCR result survive the trip to Turbopuffer?
Yes — call /v1/ocr with include_image_base64, and POMA folds the figure's description into the chunk's to_embed text, so it becomes a normal, searchable row. Images without bytes or annotation become visible, counted markers.
Why does my Turbopuffer assemble() call return empty results after a Mistral OCR ingest?
Almost always a missing query flag: Turbopuffer doesn't return attributes by default, so assemble() has nothing to key off unless the query passes include_attributes=["file_id","chunkset_index"].
Related recipes
Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Pinecone · Mistral OCR → Weaviate
Same store, different parser: browse all pipeline recipes — Turbopuffer combo pages for other parsers ship alongside this one.
Foundations: The Optimal Chunker for Mistral OCR · Turbopuffer Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide