Mistral OCR to Pinecone: Chunking for RAG
The short answer: Mistral OCR gives you excellent per-page markdown; Pinecone gives you excellent serverless filtered ANN with namespaces. Wired together naively — flatten, split, embed, stuff everything into metadata — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy and Pinecone's ~40 KB metadata cap punishes the workaround. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON (auto-detected), emits hierarchy-preserving chunksets, and you upsert their embeddings with compact {file_id, page, depth} metadata — full text stays in your doc store or the .poma archive.
What each end of the pipeline actually provides
Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, image bytes inline when you set include_image_base64. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for Mistral OCR.
Pinecone provides serverless indexes, namespaces for hard tenant partitioning, metadata filtering ($eq, $in, $gte, …) under a ~40 KB per-vector metadata limit, and sparse-dense hybrid support. What it doesn't provide: any opinion about what a vector should represent, or a place for long text — metadata is a filter mechanism, not a document store. Details: The Optimal Chunks for the Best Retrieval in Pinecone.
The naive wiring, and where it breaks
The common recipe — "\n".join(p["markdown"] for p in pages) → RecursiveCharacterTextSplitter → embed → index.upsert(...) — breaks in a pair-specific way:
- Mistral's page indices vanish at the join, so no metadata field can cite a page and
$gte/$ltepage filters have nothing to act on. - The metadata cap defeats the usual fix. To make context-free fragments usable, teams stuff the full chunk text plus breadcrumb lineage into metadata — and hit Pinecone's ~40 KB cap on exactly the long, deeply nested sections Mistral extracts best. Then comes silent trimming, and the lineage is gone again.
- Overlap inflates the namespace. Splitter overlap embeds every boundary span twice, so top-k results arrive as near-duplicates that crowd out the passage that actually answers the question.
Chunksets fix this at the unit level: each is a self-explanatory root-to-leaf path, so the vector needs only compact filter metadata, not smuggled context. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install mistralai poma pinecone sentence-transformersimport os
from mistralai import Mistral
from pinecone import Pinecone, ServerlessSpec
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer
# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
model="mistral-ocr-latest",
document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
f.write(ocr.model_dump_json())
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.mistral-ocr.json", # Mistral shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive — it carries the page numbers
)
# Page numbers live on the archive's chunk records; the SDK's typed objects
# expose depth but not page. Look both up by chunk index.
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
page_of = {c["chunk_index"]: c.get("page") for c in archive["chunks"]}
depth_of = {c.chunk_index: c.depth for c in result.chunks}
# 3. Embed chunksets; upsert with COMPACT metadata (text lives outside Pinecone).
model = SentenceTransformer("all-MiniLM-L6-v2")
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
pc.create_index(
name="contracts", dimension=384, metric="cosine",
spec=ServerlessSpec(cloud="aws", region="us-east-1"),
)
index = pc.Index("contracts")
file_id = result.chunks[0].file_id
store = {} # your doc store — or resolve IDs from the .poma archive
vectors = []
for cs in result.chunksets:
leaf = cs.chunks[-1] # chunk indices in document order; the last one is the leaf
vec_id = f"{file_id}:{cs.chunkset_index}"
store[vec_id] = cs.to_embed
vectors.append({
"id": vec_id,
"values": model.encode(cs.to_embed).tolist(),
"metadata": {"file_id": file_id, "page": page_of[leaf],
"depth": depth_of[leaf], "chunkset_index": cs.chunkset_index},
})
index.upsert(vectors=vectors, namespace="acme-corp")
# 4. Filtered query, then cheatsheet assembly from the doc store.
q = model.encode("What are the early termination conditions?").tolist()
res = index.query(
vector=q, top_k=3, namespace="acme-corp",
filter={"file_id": {"$eq": file_id}}, include_metadata=True,
)
cheatsheet = "\n\n".join(store[m["id"]] for m in res["matches"])POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases (base64 → described; annotation → spliced; neither → visible marker, counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "mistral" or "none".
Metadata mapping: POMA fields → Pinecone primitives
| POMA chunk field | Pinecone primitive | What it enables |
|---|---|---|
to_embed | dense values (your embedding model) | paraphrase retrieval |
to_embed tokens | optional sparse values on the same record | exact-term hybrid retrieval |
file_id | metadata, $eq filter | scope queries to one document |
page (on the archive's chunk records, derived from Mistral pages[].index; take the chunkset's leaf chunk) | metadata, $gte/$lte filters | page-cited answers, page-range scoping |
depth (of the chunkset's leaf chunk) | metadata | filter/re-rank by hierarchy level |
chunkset_index | metadata | stable ordering at assembly time |
| chunkset text + lineage | your doc store or .poma archive — not metadata | cheatsheet assembly under the ~40 KB cap |
Frequently asked questions
How do I get Mistral OCR results into Pinecone for RAG?
Save the raw /v1/ocr JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), embed each chunkset's to_embed with your model, and upsert to a serverless index with compact {file_id, page, depth} metadata. Full text stays in your doc store or the .poma archive.
Why does Pinecone's 40 KB metadata limit matter for Mistral OCR chunks?
Stuffing full chunk text plus lineage into metadata — the usual fix for context-free fragments — hits the ~40 KB cap on exactly the long, deeply nested sections Mistral extracts best. Keep metadata compact and resolve retrieved IDs to text outside Pinecone.
What Pinecone namespace strategy fits Mistral OCR ingestion?
One namespace per tenant or corpus; a file_id metadata filter scopes to one document inside it. POMA writes file_id on every chunk and chunkset, so the two compose cleanly.
How do I keep Mistral OCR page numbers for citations in Pinecone?
Never join pages[] before chunking. POMA records pages[].index as page on every chunk in the .poma archive; read it from the archive's chunk records (take the chunkset's leaf chunk) and the page number lands in metadata where numeric filters and citations can use it — a join-then-split pipeline destroys it irrecoverably.
Does sparse-dense hybrid search in Pinecone help with OCR'd documents?
Yes — clause numbers, invoice IDs, and defined terms are exact tokens dense embeddings blur. But sparse values only help when those tokens survive intact inside self-explanatory chunksets, not smeared across overlapping fragments.
Related recipes
Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Weaviate · Mistral OCR → pgvector
Same store, different parser: LlamaParse → Pinecone · Textract → Pinecone · Docling → Pinecone
Foundations: The Optimal Chunker for Mistral OCR · The Optimal Chunks for Pinecone · All pipeline recipes · RAG chunking guide