Skip to content

Mistral OCR to OpenSearch: Chunking for RAG ​

The short answer: Mistral OCR gives you excellent per-page markdown; OpenSearch gives you its own k-NN plugin plus a dedicated hybrid query type built independently since forking from Elasticsearch in 2021. Wired together naively — flatten, split, index — you lose the file_id/page context a filter would need right when OpenSearch's hybrid query most needs one to bound a fused, normalized ranking. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON, emits hierarchy-preserving chunksets, and records_from_archive() turns them into documents keyed by a deterministic chunkset_uuid — content-free, with the actual text living on a volume assemble() reads back at query time.

What each end of the pipeline actually provides ​

Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, plus inline image bytes when you set include_image_base64. What it doesn't provide: cross-page hierarchy or retrieval units. Details: The Optimal Chunker for Mistral OCR.

OpenSearch provides knn_vector fields backed by HNSW via nmslib or faiss, native BM25 on analyzed text fields, and a dedicated hybrid query type that fuses lexical and vector sub-queries through a search pipeline's normalization processor — a distinct mechanism from Elasticsearch's rank RRF. What it doesn't provide: any opinion about what a document's fields should contain, or protection against an unbounded fused query. Details: OpenSearch Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — "\n".join(p["markdown"] for p in pages) → fixed-window splitter → osc.bulk(...) with a text field and a knn_vector field — breaks in a pair-specific way:

  • Mistral's page and file context vanish at the join, leaving no file_id field to filter on when a hybrid query needs one to scope its search.
  • OpenSearch's hybrid fusion needs a configured pipeline, not just two queries. Unlike Elasticsearch's automatic rank RRF, hybrid only produces a sensible combined score once a search pipeline with a normalization processor is set up. Naive pipelines run it before that step exists, or run it unbounded because there's no file_id filter to attach.
  • Overlap-inflated near-duplicates dominate the normalized ranking. With no document-level filter to bound the query, near-duplicate fragments from splitter overlap crowd the fused top-k results across the whole corpus rather than within one document.

The pipeline, end to end ​

bash
pip install mistralai opensearch-py
python
import os
from mistralai.client import Mistral  # mistralai>=2 is a namespace package; the client lives in mistralai.client
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from opensearchpy import OpenSearch

# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
    model="mistral-ocr-latest",
    document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
    include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
    f.write(ocr.model_dump_json())

# 2. Chunk the raw result and keep the .poma archive — the volume's source of truth.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.mistral-ocr.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
osc = OpenSearch(hosts=[{"host": "localhost", "port": 9200}])

osc.indices.create(index="poma", body={"settings": {"index.knn": True},  # knn_vector fields need index.knn=true, otherwise the mapping is rejected (and ignore=400 hides it)
                                       "mappings": {"properties": {
    "embedding": {"type": "knn_vector", "dimension": embedder.dims,
                  "method": {"name": "hnsw", "space_type": "cosinesimil", "engine": "faiss"}},
    "file_id": {"type": "keyword"},
    "chunkset_index": {"type": "integer"},
}}}, ignore=400)

# 3. Content-free ingest — the document holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    osc.index(index="poma", id=r.id, body={
        "embedding": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
    })

# 4. Retrieval — k-NN search, full document returned by default, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
osc.indices.refresh(index="poma")  # make the just-indexed docs searchable before querying
res = osc.search(index="poma", body={"query": {"knn": {"embedding": {
    "vector": qv, "k": 10}}}})
context = assemble(res, volume=vol)  # -> [{"file_id", "content"}, ...]

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. The document id — chunkset_uuid(file_id, chunkset_index) — is deterministic, so re-indexing the same document never mints duplicates, and file_id remains available to bound any hybrid query you later configure a normalization pipeline for.

Metadata mapping: POMA fields → OpenSearch primitives ​

POMA fieldOpenSearch primitiveWhat it enables
to_embed (via embedder.embed)knn_vector fieldHNSW-backed k-NN ranking, fusable via the hybrid query type
chunkset_uuid(file_id, chunkset_index)document _iddeterministic — same chunkset, same document, across re-indexing
file_idkeyword fieldbool.filter scoping to one document, bounds hybrid queries
chunkset_indexinteger field, returned by defaultrequired by assemble(), no extra flag needed
chunk content, page/depth lineagenot stored in the document — lives on the Volumefetched by assemble() at retrieval, deduplicated into a cheatsheet

Frequently asked questions ​

How do I get Mistral OCR results into OpenSearch for RAG? ​

Save the raw /v1/ocr JSON, run PrimeCut().ingest() with download_dir/filename to keep the .poma archive, turn it into records with records_from_archive(), and index each as a document keyed by chunkset_uuid. Run a knn search and pass the response to assemble() — no extra metadata flag needed.

Why not just split Mistral's markdown and index it straight into OpenSearch? ​

Flattening discards the file_id/page context a filter needs, and OpenSearch's hybrid query type only fuses lexical and vector scores sensibly once a search pipeline's normalization processor is configured — run it unbounded against context-free fragments and overlap-inflated duplicates dominate the ranking.

How should a Mistral OCR chunkset be mapped into an OpenSearch index? ​

embedding as knn_vector (dimension matching your embedder, HNSW via faiss/nmslib), file_id as keyword, chunkset_index as integer — one document per chunkset, content itself living on the volume under the content-free pattern.

Do images in the Mistral OCR result survive the trip to OpenSearch? ​

Yes — call /v1/ocr with include_image_base64, and POMA folds the figure's description into the chunk's to_embed text, so it becomes part of a normal, searchable document. Images without bytes or annotation become visible, counted markers.

Is the retrieval code the same for OpenSearch as for Elasticsearch? ​

The knn query shape is close enough that assemble() duck-types both, but the schema (knn_vector vs dense_vector) and hybrid mechanics (search pipeline vs rank RRF) differ — check your specific version rather than assuming interchangeability.

Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Weaviate · Mistral OCR → Milvus

Same store, different parser: browse all pipeline recipes — OpenSearch combo pages for other parsers ship alongside this one. Running the upstream project instead? See Mistral OCR → Elasticsearch.

Foundations: The Optimal Chunker for Mistral OCR · OpenSearch Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide