Mistral OCR to Milvus: Chunking for RAG
The short answer: Mistral OCR gives you excellent per-page markdown; Milvus gives you filtered vector search at serious scale — partitions, scalar filter expressions, and your pick of HNSW, IVF, or DiskANN. Wired together naively — flatten, split, embed, insert — the pipeline underuses both, because nothing in between rebuilds the document hierarchy or populates the scalar fields Milvus filters on. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON (auto-detected), emits hierarchy-preserving chunksets, and each chunkset lands in Milvus with file_id, page (read from the archive's chunk records), depth, and chunkset_index as scalar fields ready for filter expressions and partition-key routing.
What each end of the pipeline actually provides
Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, image bytes inline when you set include_image_base64. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for Mistral OCR.
Milvus provides collections with typed schemas, partitions with automatic partition-key routing for multi-tenancy, scalar fields queried through filter expressions (file_id == "...", page <= 10), index types from in-memory HNSW through IVF to disk-resident DiskANN, and — in recent versions — sparse vectors with built-in BM25 for hybrid search. What it doesn't provide: any opinion about what a row should contain. Empty scalar fields filter nothing; fragment vectors retrieve fragments. Details: The Optimal Chunks for the Best Retrieval in Milvus.
The naive wiring, and where it breaks
The common recipe — "\n".join(p["markdown"] for p in pages) → RecursiveCharacterTextSplitter → embed → client.insert(...) — breaks in a pair-specific way:
- The scalar fields end up empty. A flat splitter emits anonymous text windows: no page (Mistral's
pages[].indexvanished at the join), no depth (heading levels discarded), often not even a reliable document ID. You definefile_id,page, anddepthin the schema, but there is nothing to put in them — so Milvus's headline feature, filtered search, has nothing to filter on, and the partition key can't route multi-tenant traffic. - Overlap bloats the index. Splitter overlap (typically 10–20%) embeds every boundary span twice. That inflates vector count across the board, and it hits DiskANN hardest: a disk-resident index pays for near-duplicate vectors in build time, disk footprint, and top-k lists where hits 2 and 3 restate hit 1.
- Retrieved fragments arrive without lineage, so the LLM answers out of context — the failure neither Mistral's OCR quality nor Milvus's recall can fix.
POMA's chunksets fix all three at the source: every chunkset is a self-explanatory root-to-leaf unit carrying its hierarchy metadata, with no overlap. On a notoriously hard reference legal document, chunksets plus cheatsheet assembly delivered the answer in 337 tokens of retrieved context versus 1,542 for a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install mistralai poma pymilvus sentence-transformersimport os
from mistralai import Mistral
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from pymilvus import DataType, MilvusClient
from sentence_transformers import SentenceTransformer
# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
model="mistral-ocr-latest",
document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
f.write(ocr.model_dump_json())
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.mistral-ocr.json", # Mistral shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive — it carries the page numbers
)
# Page numbers live on the archive's chunk records; the SDK's typed objects
# expose depth but not page. Look both up by chunk index.
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
page_of = {c["chunk_index"]: (c.get("page") if c.get("page") is not None else -1)
for c in archive["chunks"]} # INT64 fields cannot take NULL
depth_of = {c.chunk_index: c.depth for c in result.chunks}
# 3. Milvus — a schema whose scalar fields mirror the hierarchy metadata.
client = MilvusClient(uri=os.environ["MILVUS_URI"])
schema = MilvusClient.create_schema()
schema.add_field("pk", DataType.INT64, is_primary=True, auto_id=True)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=384)
schema.add_field("text", DataType.VARCHAR, max_length=65535)
schema.add_field("file_id", DataType.VARCHAR, max_length=256, is_partition_key=True)
schema.add_field("page", DataType.INT64)
schema.add_field("depth", DataType.INT64)
schema.add_field("chunkset_index", DataType.INT64)
index_params = client.prepare_index_params()
index_params.add_index(field_name="vector", index_type="HNSW", metric_type="COSINE")
client.create_collection("contracts", schema=schema, index_params=index_params)
# 4. Embed chunksets and insert — hierarchy metadata rides along as scalars.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
client.insert(
collection_name="contracts",
data=[
{
"vector": model.encode(cs.to_embed).tolist(),
"text": cs.to_embed,
"file_id": cs.file_id,
# cs.chunks holds chunk indices in document order; the last is the leaf.
"page": page_of[cs.chunks[-1]],
"depth": depth_of[cs.chunks[-1]],
"chunkset_index": cs.chunkset_index,
}
for cs in result.chunksets
],
)
# 5. Filtered search — scope to one document, then assemble the cheatsheet.
query = "What are the early termination conditions?"
hits = client.search(
collection_name="contracts",
data=[model.encode(query).tolist()],
limit=5,
filter='file_id == "contract.mistral-ocr.json"',
output_fields=["text", "page", "depth", "chunkset_index"],
)
ordered = sorted(hits[0], key=lambda h: h["entity"]["chunkset_index"])
cheatsheet = "\n\n".join(dict.fromkeys(h["entity"]["text"] for h in ordered))POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases (base64 → described; annotation → spliced; neither → visible marker, counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "mistral" or "none". The final step is the cheatsheet: retrieved chunksets are deduplicated (shared ancestor headings appear once) and merged into a single prompt-ready block — Milvus returns the rows, your client assembles them.
Metadata mapping: POMA fields → Milvus primitives
| POMA chunk field | Milvus primitive | What it enables |
|---|---|---|
to_embed | FLOAT_VECTOR field (add a sparse/BM25 field for hybrid) | paraphrase + exact-term retrieval |
file_id | VARCHAR scalar field, partition key | per-document filters, automatic multi-tenant routing |
page (on the archive's chunk records, derived from Mistral pages[].index; take the chunkset's leaf chunk) | INT64 scalar field | page-cited answers, page <= N filter expressions |
depth (of the chunkset's leaf chunk) | INT64 scalar field | filter/re-rank by hierarchy level |
chunkset_index | INT64 scalar field | stable ordering at cheatsheet assembly |
| chunkset text | VARCHAR field in output_fields | assembly without a second store |
Frequently asked questions
How do I get Mistral OCR results into Milvus for RAG?
Save the raw /v1/ocr JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), embed each chunkset's to_embed, and insert rows whose scalar fields carry file_id, page, depth, and chunkset_index — page comes from the archive's chunk records (the chunkset's leaf chunk), depth from the matching typed chunk. Retrieve with a filtered search and assemble cheatsheets client-side.
What scalar fields should a Milvus collection have for Mistral OCR chunks?
file_id (VARCHAR, the natural partition-key candidate), page (INT64, recorded on every chunk in the .poma archive and derived from Mistral's pages[].index), depth (INT64), and chunkset_index (INT64) alongside the vector and text. They power filter expressions and ordering at assembly time.
Should I use a Milvus partition key for multi-tenant Mistral OCR content?
Yes — declare file_id (or a tenant ID) as the partition key so filtered searches only touch matching partitions. POMA populates file_id on every chunkset, so the key always has a value to route on.
Which Milvus index type fits Mistral OCR chunksets: HNSW, IVF, or DiskANN?
All three work; choose by corpus size and latency. Chunking controls vector count: overlap-free chunksets embed each span once, which keeps any index smaller and matters most for DiskANN's disk footprint.
Does Milvus hybrid sparse plus BM25 search help with Mistral OCR documents?
Yes — OCR'd documents carry exact tokens (clause numbers, IDs, part codes) that dense embeddings blur. Recent Milvus versions store sparse BM25 vectors next to dense ones and fuse both at query time.
Related recipes
Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Chroma · Mistral OCR → Weaviate
Same store, different parser: Unstructured.io → Milvus · Marker → Milvus · Azure Document Intelligence → Milvus
Foundations: The Optimal Chunker for Mistral OCR · The Optimal Chunks for Milvus · All pipeline recipes · RAG chunking guide