Source: http://www.poma-ai.com/docs/optimal-chunker-mistral-ocr

# Mistral OCR Chunking for RAG: From OCR Output to Retrieval Units

<ByAuthor />

"Mistral OCR chunking" means two different things, and most searches for it land on the wrong one.

1. **Chunking the input.** Mistral OCR has per-request size and page limits. Mistral's [Document Chunking cookbook](https://docs.mistral.ai/cookbooks/mistral-ocr-documentchunking-readme) splits an oversized PDF into smaller files and calls the endpoint for each. That is a pre-processing step for API limits.
2. **Chunking the output for RAG.** Once you have the `/v1/ocr` result, something has to decide what each embedding vector represents. Mistral leaves that entirely to you, and it is where retrieval quality is won or lost.

This page is about the second. It shows what Mistral OCR returns, why the usual text splitters waste most of it, and a complete pipeline from the OCR call to retrieval-ready chunks that keep their page numbers and heading context. If your files exceed the API limits, run Mistral's cookbook first, then continue here with the results.

## Quick start: from `/v1/ocr` to chunksets

Mistral OCR chunking for RAG in three lines: run the OCR exactly as you do today, save the raw JSON, and hand it to PrimeCut. The Mistral shape is auto-detected.

```python
import os
from mistralai.client import Mistral  # mistralai>=2 is a namespace package; the client lives in mistralai.client
from poma import PrimeCut

# 1. Your Mistral OCR call, unchanged. Save the JSON, not just the markdown.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
    model="mistral-ocr-latest",
    document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
    include_image_base64=True,  # lets PrimeCut describe figures instead of dropping them
)
with open("contract.mistral-ocr.json", "w") as f:
    f.write(ocr.model_dump_json())

# 2. Chunk the raw result. Mistral OCR is skipped, only the chunking runs.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.mistral-ocr.json")

print(len(result.chunks), len(result.chunksets))
print(result.chunksets[0].to_embed)  # embedding-ready text with its heading path
```

From `result.chunks` and `result.chunksets` onward the stack is yours. The [pipeline recipes](#ship-the-chunks-to-your-vector-database) below wire the result into thirteen vector databases; the [end-to-end example](#the-mistral-ocr-rag-pipeline-end-to-end) further down uses Qdrant.

## What Mistral OCR returns, and where it stops

Mistral OCR (`mistral-ocr-latest`, [OCR 4 since June 2026](https://mistral.ai/news/ocr-4/)) returns a JSON envelope with one entry per page:

```json
{
  "model": "mistral-ocr-latest",
  "pages": [
    {
      "index": 0,
      "markdown": "# Annual Report 2025\n\n## Financial highlights\n\n…",
      "images": [{ "id": "img-0.jpeg", "image_base64": "…" }],
      "tables": [],
      "header": null,
      "footer": null,
      "dimensions": { "…": "…" },
      "hyperlinks": [],
      "blocks": null,
      "confidence_scores": null
    }
  ],
  "document_annotation": null,
  "usage_info": { "pages_processed": 1 }
}
```

Within a page, structure is recovered: headings become `#`/`##` markdown, lists stay lists, tables arrive inline or in the `tables` side-channel (`table_format` selects markdown or HTML). OCR 4 adds typed `blocks` with bounding boxes when you pass `include_blocks=True` and per-page, per-block or per-word `confidence_scores`. The API also exposes `extract_header`/`extract_footer`, which move running headers and footers out of the markdown into their own fields. That is real, hard-won structure, and it is exactly the part most pipelines then throw away.

What Mistral OCR deliberately does **not** do:

- **Cross-page hierarchy.** Each page is independent. A `## Termination clauses` heading on page 41 has no machine-readable link to the `# Master Services Agreement` chapter that opened on page 3. OCR 4 blocks are typed and located, but still per-page and flat.
- **Chunking for retrieval.** The response is display-ready markdown, not retrieval units. Which sentences travel together into one embedding is your decision.
- **Image semantics.** Depending on your call, images come as base64 bytes, as a precomputed `image_annotation`, or as bare references with no content at all. Something downstream has to make figures searchable, or at least make their loss visible.

## Why generic text splitters waste Mistral OCR output

The default move, concatenate `pages[].markdown` into one string and run `RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)`, destroys most of what you paid for:

| What Mistral recovered | What a character splitter does with it |
| --- | --- |
| Heading levels (`#`, `##`, `###`) | Ignored; cuts fall wherever the character count lands |
| Tables (inline or `tables[]`) | Sliced mid-row; header row separated from data rows |
| Page boundaries (`pages[].index`) | Lost at concatenation; chunks cannot cite a page |
| Reading order across columns | Preserved only by luck of the cut positions |
| Figures | Base64 blobs either bloat a chunk or get regex-stripped silently |
| OCR 4 blocks and confidence scores | Never looked at |

Markdown-header splitters, and block-aware splitters built on OCR 4 output, are a real step up: cuts land on structural boundaries. They still produce **isolated fragments**. The chunk under `### Early termination` no longer knows it lives inside `## Termination clauses` inside `# Master Services Agreement`. At retrieval time the model reads an orphaned paragraph and answers out of context. That failure, not OCR accuracy, is commonly where Mistral OCR RAG pipelines lose quality. The [chunking strategies guide](/rag-chunking-strategies-text-splitters) covers the full taxonomy.

## Hierarchy-aware chunking on the raw result

PrimeCut is an ingestion and chunking engine: given a PDF it does the parsing itself. Its bring-your-own-OCR path is for teams that already run Mistral. It consumes the unmodified `/v1/ocr` response and runs only the downstream stages on it:

1. **Shape validation up front.** The payload is fingerprinted (`pages[]` carrying `markdown` and `index`). A corrupted or mislabeled upload fails immediately with a clear 422, never a silently degraded index.
2. **All three image cases.** `image_base64` is decoded and described, so figures become searchable text. An existing `image_annotation` is spliced in as the description. Neither present: an inert positional marker, counted in `content_metadata`, so the loss is visible.
3. **Table splicing.** Side-channel `tables[]` content is spliced at its reference, HTML or markdown. Rows and columns survive into the chunk layer, so an embedding never sees half a table.
4. **Noise removal.** Running headers and footers that repeat on every page are stripped from the markdown. The separate `header`/`footer` fields from `extract_header`/`extract_footer` are ignored, so keep real headings in the page body.
5. **Heading enrichment.** Where the markdown under-marks headings, common in scanned or design-heavy documents, headings are restored before chunking.
6. **Hierarchy to chunks and chunksets.** The cross-page heading tree is rebuilt. Every sentence becomes a chunk record with its `depth`, its `page` (in the `.poma` archive's chunk records; the Python SDK's chunk objects expose `depth` but not yet `page`) and an embedding-ready `to_embed` form, grouped into [chunksets](/learn/chunking/chunksets): root-to-leaf paths that keep each retrieved passage self-explanatory.

Because the OCR already ran, you pay for the chunking only; PrimeCut's own OCR front-end is skipped. To force or suppress detection, set `external_ocr_source` to `"mistral"` or `"none"` on the ingest request; the API also exposes `chunk_external_ocr_result` for explicit routing ([API reference](https://api.poma-ai.com/v3/docs)).

One illustration of what the hierarchy is worth: on a legal document in our reference set, the same question was answered from 337 tokens of retrieved context with chunksets versus 1,542 with a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the [ingestion guide](/document-ingestion-chunking-rag) has the broader numbers.

## The Mistral OCR RAG pipeline, end to end

The quick start stops at chunksets. This version continues into a vector store and back out, using Qdrant; the same shape applies to every store in the [recipes](#ship-the-chunks-to-your-vector-database).

```bash
pip install mistralai 'poma[qdrant]'
```

```python
import os
from mistralai import Mistral
from poma import PrimeCut
from poma.integrations.qdrant import PomaQdrant

# 1. Mistral OCR, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
    model="mistral-ocr-latest",
    document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
    include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
    f.write(ocr.model_dump_json())

# 2. Chunk: raw result JSON in, chunks and chunksets out.
poma = PrimeCut()
result = poma.ingest("contract.mistral-ocr.json")

# 3. Embed and store: hybrid points with hierarchy payloads, one call.
qdrant = PomaQdrant(
    url=os.environ["QDRANT_URL"],
    api_key=os.environ["QDRANT_API_KEY"],
    cloud_inference=True,
    collection_name="contracts",
    dense_model="sentence-transformers/all-MiniLM-L6-v2",
    sparse_model="Qdrant/bm25",
    dense_size=384,
    auto_create_collection=True,
)
qdrant.upsert_poma_points(result)

# 4. Retrieve prompt-ready context: hits merged along their shared heading path.
cheatsheets = qdrant.get_cheatsheets(query="What are the early termination conditions?", limit=3)
print(cheatsheets[0]["content"])
```

Page numbers from `pages[].index` are kept on every chunk record in the `.poma` archive (`client.ingest(..., download_dir="store", filename="contract.poma")` keeps it; `poma.utils.unpack_poma_archive` reads it), so retrieved context can cite the page it came from.

## Chunking options for Mistral OCR output, compared

| Approach | Structure used | Cross-page hierarchy | Tables | Images | Page numbers kept |
| --- | --- | --- | --- | --- | --- |
| Recursive character splitter | None | ✗ | Cut mid-row | Stripped or inlined as noise | ✗ |
| Markdown-header splitter | Per-page headings | ✗ | Usually survive | Usually stripped | Manual |
| Block-aware splitter (OCR 4 `blocks`) | Typed blocks per page | ✗ | Survive | Manual | Manual |
| Semantic chunker | Embedding similarity | ✗ | Fragile | Stripped | Manual |
| **PrimeCut on the raw result** | **Full heading tree, rebuilt across pages** | **✓ chunksets** | **Spliced, never cut** | **Described or visibly counted** | **✓ per chunk record (archive)** |

## Frequently asked questions

### How do I chunk Mistral OCR output for RAG?

Save the raw /v1/ocr JSON, not a flattened string, and hand it to a structure-aware chunker. PrimeCut auto-detects the Mistral shape, rebuilds the heading hierarchy across pages, keeps tables whole, describes or visibly counts images, and returns chunks plus chunksets ready to embed. Do not concatenate pages[].markdown and run a character splitter over it: that throws away the page boundaries and headings Mistral already recovered.

### Does Mistral OCR chunk documents itself?

No. Mistral OCR returns display-ready markdown per page, plus tables, images and, since OCR 4, typed blocks with bounding boxes and confidence scores. Deciding what one embedding vector should represent is left to you. Mistral's "Document Chunking" cookbook is about a different problem: splitting a PDF that exceeds the API's size or page limit into smaller files before calling the model.

### What is Mistral's Document Chunking cookbook for?

It splits oversized documents into pieces that fit Mistral OCR's per-request size and page limits and calls the endpoint asynchronously for each piece. It is a pre-processing utility for API constraints, not a retrieval chunking strategy. Use it if your files are too large for one call, then chunk the merged results for RAG as described on this page.

### Mistral OCR to markdown or Mistral OCR to JSON: which should I save?

Save the JSON. The markdown alone loses page indexes, image references, table side-channels, header and footer fields, and the OCR 4 blocks and confidence scores. PrimeCut reads the JSON envelope directly and keeps page numbers on every chunk, which is what lets answers cite pages later.

### Can I use Mistral OCR 4 blocks and bounding boxes for chunking?

Typed blocks (title, table, equation, signature and so on) are better cut boundaries than character counts, and a block-aware splitter is a real step up from a text splitter. They are still per-page and flat: a title block on page 41 does not know which chapter it belongs to. PrimeCut rebuilds that cross-page hierarchy from the markdown and table content and returns each sentence with its full heading path.

### Can I use Mistral OCR results with PrimeCut without re-running OCR?

Yes. Upload the raw result JSON and PrimeCut skips its own OCR front-end and runs only the downstream structure, chunking and retrieval-preparation stages. You pay for chunking, not for OCR you already paid Mistral for.

### What happens to images in a Mistral OCR result during chunking?

Three cases are handled: image_base64 present, the figure is decoded and described so it becomes searchable text; an image_annotation present, the annotation is spliced in as the description; neither present, the reference is replaced by an inert marker and counted in content_metadata, so the loss is visible rather than silent.

### Should I switch OCR engines to fix retrieval quality?

Usually not. If Mistral OCR fits your cost, latency or data-locality constraints, keep it. Retrieval quality is decided mostly by chunking and by what context each retrieved unit carries, not by swapping one strong OCR engine for another.

## Ship the chunks to your vector database

End-to-end recipes for wiring Mistral OCR into the store you already run:

- [Mistral OCR → Chroma](/pipelines/mistral-ocr-to-chroma)
- [Mistral OCR → Elasticsearch](/pipelines/mistral-ocr-to-elasticsearch)
- [Mistral OCR → LanceDB](/pipelines/mistral-ocr-to-lancedb)
- [Mistral OCR → Milvus](/pipelines/mistral-ocr-to-milvus)
- [Mistral OCR → MongoDB Atlas](/pipelines/mistral-ocr-to-mongodb-atlas)
- [Mistral OCR → OpenSearch](/pipelines/mistral-ocr-to-opensearch)
- [Mistral OCR → pgvector](/pipelines/mistral-ocr-to-pgvector)
- [Mistral OCR → Pinecone](/pipelines/mistral-ocr-to-pinecone)
- [Mistral OCR → Qdrant](/pipelines/mistral-ocr-to-qdrant)
- [Mistral OCR → Redis](/pipelines/mistral-ocr-to-redis)
- [Mistral OCR → Turbopuffer](/pipelines/mistral-ocr-to-turbopuffer)
- [Mistral OCR → Vespa](/pipelines/mistral-ocr-to-vespa)
- [Mistral OCR → Weaviate](/pipelines/mistral-ocr-to-weaviate)

Or browse [all pipeline recipes](/pipelines/).

## The rest of the series

The optimal chunker, for every OCR and parsing tool you already run:

- [AWS Textract](/optimal-chunker-textract)
- [Azure Document Intelligence](/optimal-chunker-azure-document-intelligence)
- [DeepSeek-OCR](/optimal-chunker-deepseek-ocr)
- [Docling](/optimal-chunker-docling)
- [LlamaParse](/optimal-chunker-llamaparse)
- [Marker](/optimal-chunker-marker)
- [MinerU](/optimal-chunker-mineru)
- [PaddleOCR-VL](/optimal-chunker-paddleocr-vl)
- [Unstructured.io](/optimal-chunker-unstructured)

Or start with the fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [Document ingestion guide](/guides/document-ingestion/) · [POMA chunksets](/learn/chunking/chunksets).