Skip to content

Mistral OCR to Weaviate: Chunking for RAG ​

The short answer: Mistral OCR gives you excellent per-page markdown; Weaviate gives you built-in hybrid search — BM25F plus vectors, blended by alpha — over filterable properties. Wired together naively — flatten, split, embed — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy, and both halves of the hybrid suffer for it. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON (auto-detected), emits hierarchy-preserving chunksets, and each becomes one Weaviate object with file_id/page/depth properties and your embedding as its vector.

What each end of the pipeline actually provides ​

Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, image bytes inline when you set include_image_base64. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for Mistral OCR.

Weaviate provides collections with named vectors, built-in hybrid search (BM25F + vector, blended by alpha), filterable properties, cross-references between collections, and generative/reranker modules. What it doesn't provide: any opinion about what an object should contain. BM25F scores whatever text you stored; the vector index ranks whatever you embedded — including context-free fragments. Details: The Optimal Chunks for the Best Retrieval in Weaviate.

The naive wiring, and where it breaks ​

The common recipe — "\n".join(p["markdown"] for p in pages) → RecursiveCharacterTextSplitter → embed → data.insert(...) — breaks in a pair-specific way:

  • The page property dies at the join. Mistral's pages[].index is gone before splitting starts, so the one property Weaviate needs for page filters and page-cited answers holds nothing truthful.
  • Hybrid's keyword half is wasted on fragments. Weaviate's BM25F is exactly right for OCR'd documents full of clause numbers and defined terms — but overlap splitting smears those exact tokens across fragment boundaries. The alpha knob then blends a blurred keyword signal with a blurred vector signal; no ratio fixes the units.
  • Overlap pollutes both result lists. Every boundary span exists twice, so BM25F and vector rankings each surface near-duplicates that crowd out the passage that actually answers the question.

Chunksets fix this at the unit level: self-explanatory root-to-leaf paths, no overlap, exact tokens intact. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).

The pipeline, end to end ​

bash
pip install mistralai poma weaviate-client sentence-transformers
python
import os
import weaviate
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.init import Auth
from weaviate.classes.query import Filter
from mistralai import Mistral
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer

# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
    model="mistral-ocr-latest",
    document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
    include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
    f.write(ocr.model_dump_json())

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.mistral-ocr.json",  # Mistral shape auto-detected
    download_dir="store",
    filename="contract.poma",     # keep the archive — it carries the page numbers
)

# Page numbers live on the archive's chunk records; the SDK's typed objects
# expose depth but not page. Look both up by chunk index.
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
page_of = {c["chunk_index"]: (c.get("page") if c.get("page") is not None else -1)
           for c in archive["chunks"]}  # INT properties cannot take None
depth_of = {c.chunk_index: c.depth for c in result.chunks}

# 3. One collection of chunksets, hierarchy as filterable properties.
client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)
client.collections.create(
    "Chunkset",
    # bring your own vectors (python-client < 4.16: vectorizer_config=Configure.Vectorizer.none())
    vector_config=Configure.Vectors.self_provided(),
    properties=[
        Property(name="content", data_type=DataType.TEXT),
        Property(name="file_id", data_type=DataType.TEXT),
        Property(name="page", data_type=DataType.INT),
        Property(name="depth", data_type=DataType.INT),
        Property(name="chunkset_index", data_type=DataType.INT),
    ],
)
chunksets = client.collections.get("Chunkset")

model = SentenceTransformer("all-MiniLM-L6-v2")
file_id = result.chunks[0].file_id
for cs in result.chunksets:
    leaf = cs.chunks[-1]  # chunk indices in document order; the last one is the leaf
    chunksets.data.insert(
        properties={"content": cs.to_embed, "file_id": file_id,
                    "page": page_of[leaf], "depth": depth_of[leaf],
                    "chunkset_index": cs.chunkset_index},
        vector=model.encode(cs.to_embed).tolist(),
    )

# 4. Hybrid query: BM25F on content + your vector, blended by alpha.
query = "What are the early termination conditions?"
res = chunksets.query.hybrid(
    query=query,
    vector=model.encode(query).tolist(),
    alpha=0.5,
    filters=Filter.by_property("file_id").equal(file_id),
    limit=3,
)
cheatsheet = "\n\n".join(o.properties["content"] for o in res.objects)
client.close()

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases (base64 → described; annotation → spliced; neither → visible marker, counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "mistral" or "none".

Metadata mapping: POMA fields → Weaviate primitives ​

POMA chunk fieldWeaviate primitiveWhat it enables
to_embedobject vector (bring-your-own)paraphrase retrieval
to_embed textcontent TEXT property, BM25F-indexedexact-term half of hybrid search
file_idTEXT property, Filter.by_propertyscope queries to one document
page (on the archive's chunk records, derived from Mistral pages[].index; take the chunkset's leaf chunk)INT propertypage filters, page-cited answers
depth (of the chunkset's leaf chunk)INT propertyfilter/re-rank by hierarchy level
chunkset_indexINT propertystable ordering at assembly time
chunkset lineageoptional cross-reference to a Chunk collectionchunk-level provenance without a second store

Frequently asked questions ​

How do I get Mistral OCR results into Weaviate for RAG? ​

Save the raw /v1/ocr JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), create a collection with file_id/page/depth properties, insert one object per chunkset with your own vector, and retrieve with query.hybrid.

What properties should a Weaviate collection have for Mistral OCR content? ​

content (TEXT, feeds BM25F), file_id (TEXT), page (INT, read from the archive's chunk records — take the chunkset's leaf chunk), depth (INT, the leaf chunk's depth), and chunkset_index (INT). POMA emits all of these, so the mapping is mechanical.

What hybrid alpha should I use in Weaviate for OCR'd documents? ​

Start at alpha=0.5 (0 = pure BM25F, 1 = pure vector). Exact tokens make the keyword half valuable — but only if chunking kept them intact. No blend ratio repairs fragments.

Why do page filters break when importing Mistral OCR markdown into Weaviate? ​

Joining pages[].markdown before splitting destroys page boundaries, leaving the page property nothing truthful to hold. POMA records pages[].index as page on every chunk in the .poma archive; read it from the archive's chunk records (take the chunkset's leaf chunk) and page arrives as a real filterable INT.

Should I use Weaviate cross-references to model Mistral OCR chunk hierarchy? ​

Optional. Cross-references model chunkset→chunk lineage cleanly for provenance UIs, but chunksets are already self-explanatory — retrieval needs no query-time graph traversal.

Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Pinecone · Mistral OCR → pgvector

Same store, different parser: LlamaParse → Weaviate · Azure Document Intelligence → Weaviate · Unstructured.io → Weaviate

Foundations: The Optimal Chunker for Mistral OCR · The Optimal Chunks for Weaviate · All pipeline recipes · RAG chunking guide