Skip to content

Mistral OCR to Pinecone: Chunking for RAG ​

The short answer: Mistral OCR gives you excellent per-page markdown; Pinecone gives you excellent serverless filtered ANN with namespaces. Wired together naively — flatten, split, embed, stuff everything into metadata — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy and Pinecone's ~40 KB metadata cap punishes the workaround. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON (auto-detected), emits hierarchy-preserving chunksets, and you upsert their embeddings with compact {file_id, page, depth} metadata — full text stays in your doc store or the .poma archive.

What each end of the pipeline actually provides ​

Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, image bytes inline when you set include_image_base64. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for Mistral OCR.

Pinecone provides serverless indexes, namespaces for hard tenant partitioning, metadata filtering ($eq, $in, $gte, …) under a ~40 KB per-vector metadata limit, and sparse-dense hybrid support. What it doesn't provide: any opinion about what a vector should represent, or a place for long text — metadata is a filter mechanism, not a document store. Details: The Optimal Chunks for the Best Retrieval in Pinecone.

The naive wiring, and where it breaks ​

The common recipe — "\n".join(p["markdown"] for p in pages) → RecursiveCharacterTextSplitter → embed → index.upsert(...) — breaks in a pair-specific way:

  • Mistral's page indices vanish at the join, so no metadata field can cite a page and $gte/$lte page filters have nothing to act on.
  • The metadata cap defeats the usual fix. To make context-free fragments usable, teams stuff the full chunk text plus breadcrumb lineage into metadata — and hit Pinecone's ~40 KB cap on exactly the long, deeply nested sections Mistral extracts best. Then comes silent trimming, and the lineage is gone again.
  • Overlap inflates the namespace. Splitter overlap embeds every boundary span twice, so top-k results arrive as near-duplicates that crowd out the passage that actually answers the question.

Chunksets fix this at the unit level: each is a self-explanatory root-to-leaf path, so the vector needs only compact filter metadata, not smuggled context. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).

The pipeline, end to end ​

bash
pip install mistralai poma pinecone sentence-transformers
python
import os
from mistralai import Mistral
from pinecone import Pinecone, ServerlessSpec
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer

# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
    model="mistral-ocr-latest",
    document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
    include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
    f.write(ocr.model_dump_json())

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.mistral-ocr.json",  # Mistral shape auto-detected
    download_dir="store",
    filename="contract.poma",     # keep the archive — it carries the page numbers
)

# Page numbers live on the archive's chunk records; the SDK's typed objects
# expose depth but not page. Look both up by chunk index.
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
page_of = {c["chunk_index"]: c.get("page") for c in archive["chunks"]}
depth_of = {c.chunk_index: c.depth for c in result.chunks}

# 3. Embed chunksets; upsert with COMPACT metadata (text lives outside Pinecone).
model = SentenceTransformer("all-MiniLM-L6-v2")
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
pc.create_index(
    name="contracts", dimension=384, metric="cosine",
    spec=ServerlessSpec(cloud="aws", region="us-east-1"),
)
index = pc.Index("contracts")

file_id = result.chunks[0].file_id
store = {}  # your doc store — or resolve IDs from the .poma archive
vectors = []
for cs in result.chunksets:
    leaf = cs.chunks[-1]  # chunk indices in document order; the last one is the leaf
    vec_id = f"{file_id}:{cs.chunkset_index}"
    store[vec_id] = cs.to_embed
    vectors.append({
        "id": vec_id,
        "values": model.encode(cs.to_embed).tolist(),
        "metadata": {"file_id": file_id, "page": page_of[leaf],
                     "depth": depth_of[leaf], "chunkset_index": cs.chunkset_index},
    })
index.upsert(vectors=vectors, namespace="acme-corp")

# 4. Filtered query, then cheatsheet assembly from the doc store.
q = model.encode("What are the early termination conditions?").tolist()
res = index.query(
    vector=q, top_k=3, namespace="acme-corp",
    filter={"file_id": {"$eq": file_id}}, include_metadata=True,
)
cheatsheet = "\n\n".join(store[m["id"]] for m in res["matches"])

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases (base64 → described; annotation → spliced; neither → visible marker, counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "mistral" or "none".

Metadata mapping: POMA fields → Pinecone primitives ​

POMA chunk fieldPinecone primitiveWhat it enables
to_embeddense values (your embedding model)paraphrase retrieval
to_embed tokensoptional sparse values on the same recordexact-term hybrid retrieval
file_idmetadata, $eq filterscope queries to one document
page (on the archive's chunk records, derived from Mistral pages[].index; take the chunkset's leaf chunk)metadata, $gte/$lte filterspage-cited answers, page-range scoping
depth (of the chunkset's leaf chunk)metadatafilter/re-rank by hierarchy level
chunkset_indexmetadatastable ordering at assembly time
chunkset text + lineageyour doc store or .poma archive — not metadatacheatsheet assembly under the ~40 KB cap

Frequently asked questions ​

How do I get Mistral OCR results into Pinecone for RAG? ​

Save the raw /v1/ocr JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), embed each chunkset's to_embed with your model, and upsert to a serverless index with compact {file_id, page, depth} metadata. Full text stays in your doc store or the .poma archive.

Why does Pinecone's 40 KB metadata limit matter for Mistral OCR chunks? ​

Stuffing full chunk text plus lineage into metadata — the usual fix for context-free fragments — hits the ~40 KB cap on exactly the long, deeply nested sections Mistral extracts best. Keep metadata compact and resolve retrieved IDs to text outside Pinecone.

What Pinecone namespace strategy fits Mistral OCR ingestion? ​

One namespace per tenant or corpus; a file_id metadata filter scopes to one document inside it. POMA writes file_id on every chunk and chunkset, so the two compose cleanly.

How do I keep Mistral OCR page numbers for citations in Pinecone? ​

Never join pages[] before chunking. POMA records pages[].index as page on every chunk in the .poma archive; read it from the archive's chunk records (take the chunkset's leaf chunk) and the page number lands in metadata where numeric filters and citations can use it — a join-then-split pipeline destroys it irrecoverably.

Does sparse-dense hybrid search in Pinecone help with OCR'd documents? ​

Yes — clause numbers, invoice IDs, and defined terms are exact tokens dense embeddings blur. But sparse values only help when those tokens survive intact inside self-explanatory chunksets, not smeared across overlapping fragments.

Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Weaviate · Mistral OCR → pgvector

Same store, different parser: LlamaParse → Pinecone · Textract → Pinecone · Docling → Pinecone

Foundations: The Optimal Chunker for Mistral OCR · The Optimal Chunks for Pinecone · All pipeline recipes · RAG chunking guide