Mistral OCR to Weaviate: Chunking for RAG
The short answer: Mistral OCR gives you excellent per-page markdown; Weaviate gives you built-in hybrid search — BM25F plus vectors, blended by alpha — over filterable properties. Wired together naively — flatten, split, embed — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy, and both halves of the hybrid suffer for it. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON (auto-detected), emits hierarchy-preserving chunksets, and each becomes one Weaviate object with file_id/page/depth properties and your embedding as its vector.
What each end of the pipeline actually provides
Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, image bytes inline when you set include_image_base64. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for Mistral OCR.
Weaviate provides collections with named vectors, built-in hybrid search (BM25F + vector, blended by alpha), filterable properties, cross-references between collections, and generative/reranker modules. What it doesn't provide: any opinion about what an object should contain. BM25F scores whatever text you stored; the vector index ranks whatever you embedded — including context-free fragments. Details: The Optimal Chunks for the Best Retrieval in Weaviate.
The naive wiring, and where it breaks
The common recipe — "\n".join(p["markdown"] for p in pages) → RecursiveCharacterTextSplitter → embed → data.insert(...) — breaks in a pair-specific way:
- The
pageproperty dies at the join. Mistral'spages[].indexis gone before splitting starts, so the one property Weaviate needs for page filters and page-cited answers holds nothing truthful. - Hybrid's keyword half is wasted on fragments. Weaviate's BM25F is exactly right for OCR'd documents full of clause numbers and defined terms — but overlap splitting smears those exact tokens across fragment boundaries. The
alphaknob then blends a blurred keyword signal with a blurred vector signal; no ratio fixes the units. - Overlap pollutes both result lists. Every boundary span exists twice, so BM25F and vector rankings each surface near-duplicates that crowd out the passage that actually answers the question.
Chunksets fix this at the unit level: self-explanatory root-to-leaf paths, no overlap, exact tokens intact. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install mistralai poma weaviate-client sentence-transformersimport os
import weaviate
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.init import Auth
from weaviate.classes.query import Filter
from mistralai import Mistral
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer
# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
model="mistral-ocr-latest",
document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
f.write(ocr.model_dump_json())
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.mistral-ocr.json", # Mistral shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive — it carries the page numbers
)
# Page numbers live on the archive's chunk records; the SDK's typed objects
# expose depth but not page. Look both up by chunk index.
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
page_of = {c["chunk_index"]: (c.get("page") if c.get("page") is not None else -1)
for c in archive["chunks"]} # INT properties cannot take None
depth_of = {c.chunk_index: c.depth for c in result.chunks}
# 3. One collection of chunksets, hierarchy as filterable properties.
client = weaviate.connect_to_weaviate_cloud(
cluster_url=os.environ["WEAVIATE_URL"],
auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)
client.collections.create(
"Chunkset",
# bring your own vectors (python-client < 4.16: vectorizer_config=Configure.Vectorizer.none())
vector_config=Configure.Vectors.self_provided(),
properties=[
Property(name="content", data_type=DataType.TEXT),
Property(name="file_id", data_type=DataType.TEXT),
Property(name="page", data_type=DataType.INT),
Property(name="depth", data_type=DataType.INT),
Property(name="chunkset_index", data_type=DataType.INT),
],
)
chunksets = client.collections.get("Chunkset")
model = SentenceTransformer("all-MiniLM-L6-v2")
file_id = result.chunks[0].file_id
for cs in result.chunksets:
leaf = cs.chunks[-1] # chunk indices in document order; the last one is the leaf
chunksets.data.insert(
properties={"content": cs.to_embed, "file_id": file_id,
"page": page_of[leaf], "depth": depth_of[leaf],
"chunkset_index": cs.chunkset_index},
vector=model.encode(cs.to_embed).tolist(),
)
# 4. Hybrid query: BM25F on content + your vector, blended by alpha.
query = "What are the early termination conditions?"
res = chunksets.query.hybrid(
query=query,
vector=model.encode(query).tolist(),
alpha=0.5,
filters=Filter.by_property("file_id").equal(file_id),
limit=3,
)
cheatsheet = "\n\n".join(o.properties["content"] for o in res.objects)
client.close()POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases (base64 → described; annotation → spliced; neither → visible marker, counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "mistral" or "none".
Metadata mapping: POMA fields → Weaviate primitives
| POMA chunk field | Weaviate primitive | What it enables |
|---|---|---|
to_embed | object vector (bring-your-own) | paraphrase retrieval |
to_embed text | content TEXT property, BM25F-indexed | exact-term half of hybrid search |
file_id | TEXT property, Filter.by_property | scope queries to one document |
page (on the archive's chunk records, derived from Mistral pages[].index; take the chunkset's leaf chunk) | INT property | page filters, page-cited answers |
depth (of the chunkset's leaf chunk) | INT property | filter/re-rank by hierarchy level |
chunkset_index | INT property | stable ordering at assembly time |
| chunkset lineage | optional cross-reference to a Chunk collection | chunk-level provenance without a second store |
Frequently asked questions
How do I get Mistral OCR results into Weaviate for RAG?
Save the raw /v1/ocr JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), create a collection with file_id/page/depth properties, insert one object per chunkset with your own vector, and retrieve with query.hybrid.
What properties should a Weaviate collection have for Mistral OCR content?
content (TEXT, feeds BM25F), file_id (TEXT), page (INT, read from the archive's chunk records — take the chunkset's leaf chunk), depth (INT, the leaf chunk's depth), and chunkset_index (INT). POMA emits all of these, so the mapping is mechanical.
What hybrid alpha should I use in Weaviate for OCR'd documents?
Start at alpha=0.5 (0 = pure BM25F, 1 = pure vector). Exact tokens make the keyword half valuable — but only if chunking kept them intact. No blend ratio repairs fragments.
Why do page filters break when importing Mistral OCR markdown into Weaviate?
Joining pages[].markdown before splitting destroys page boundaries, leaving the page property nothing truthful to hold. POMA records pages[].index as page on every chunk in the .poma archive; read it from the archive's chunk records (take the chunkset's leaf chunk) and page arrives as a real filterable INT.
Should I use Weaviate cross-references to model Mistral OCR chunk hierarchy?
Optional. Cross-references model chunkset→chunk lineage cleanly for provenance UIs, but chunksets are already self-explanatory — retrieval needs no query-time graph traversal.
Related recipes
Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Pinecone · Mistral OCR → pgvector
Same store, different parser: LlamaParse → Weaviate · Azure Document Intelligence → Weaviate · Unstructured.io → Weaviate
Foundations: The Optimal Chunker for Mistral OCR · The Optimal Chunks for Weaviate · All pipeline recipes · RAG chunking guide