Source: http://www.poma-ai.com/docs/optimal-chunker-mistral-ocr

# The Optimal Chunker for Mistral OCR

<ByAuthor />

**The short answer:** the optimal chunker for Mistral OCR is one that consumes the raw `/v1/ocr` JSON *as-is*, preserves the markdown structure Mistral recovered, rebuilds the heading hierarchy across pages, and emits retrieval units that carry their own context. POMA's bring-your-own-OCR connector does exactly this: you upload the unmodified Mistral response, and PrimeCut returns hierarchy-preserving **chunks** and **chunksets** — no re-OCR, no markdown flattening, no hand-rolled splitter.

This page explains what Mistral OCR gives you, where it stops, why generic text splitters squander it, and how to go from a `/v1/ocr` call to embedded chunksets in a few lines.

## What Mistral OCR gives you — and where it stops

Mistral OCR (`mistral-ocr-latest`, the `/v1/ocr` endpoint) is one of the strongest OCR-to-markdown engines available: you send a PDF or image, it returns **per-page markdown** in a JSON envelope:

```json
{
  "pages": [
    {
      "index": 0,
      "markdown": "# Annual Report 2025\n\n## Financial highlights\n\n…",
      "images": [{ "id": "img-0.jpeg", "image_base64": "…" }]
    }
  ]
}
```

Within each page, structure is recovered: headings become `#`/`##` markdown, lists stay lists, tables arrive inline or as side-channel content. That's real, hard-won structure — and it's exactly the part most pipelines then throw away.

What Mistral OCR deliberately does **not** do:

- **Cross-page hierarchy.** Each page's markdown is independent. A `## Termination clauses` heading on page 41 has no machine-readable link to the `# Master Services Agreement` chapter that opened on page 3.
- **Chunking.** The response is display-ready markdown, not retrieval-ready units. Deciding what an embedding vector should represent is left entirely to you.
- **Image semantics.** Depending on your call parameters, images come as base64 bytes, as pre-computed annotations, or as bare refs with no content at all. Something downstream has to make figures searchable — or at least make their loss visible.

Mistral's docs stop at "here is markdown." The gap between *markdown* and *good retrieval* is where RAG pipelines quietly fail.

## Why generic text splitters waste Mistral OCR output

The default move — concatenate `pages[].markdown` into one string, run `RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)` — destroys most of what you just paid for:

| What Mistral recovered | What a character splitter does with it |
| --- | --- |
| Heading levels (`#`, `##`, `###`) | Ignored — cuts fall wherever the character count lands |
| HTML/markdown tables | Sliced mid-row; header row separated from data rows |
| Page boundaries (`pages[].index`) | Lost at concatenation — chunks can't cite a page |
| Reading order across columns | Preserved only by luck of the cut positions |
| Figure positions | Base64 blobs either bloat a chunk or get regex-stripped silently |

Markdown-header splitters (splitting on `##` boundaries) are a step up, but they still produce **isolated fragments**: the chunk under `### Early termination` no longer knows it lives inside `## Termination clauses` inside `# Master Services Agreement`. At retrieval time the LLM reads an orphaned paragraph and answers out of context. That failure mode — not OCR accuracy — is where most Mistral-OCR-based RAG pipelines lose their quality.

For the full taxonomy of these failure modes, see the [RAG chunking strategies guide](/rag-chunking-strategies-text-splitters).

## The optimal chunker: hierarchy-aware chunking on the raw result

POMA's **bring-your-own-OCR** (BYOCR) connector was built for teams who already run their own OCR step. Mistral is the original BYOCR source — the connector consumes the *unmodified* `/v1/ocr` response and runs only POMA's downstream stages on it:

1. **Shape validation, up front.** The payload is fingerprinted (`pages[]` with `markdown` + `index` keys). A corrupted or mislabeled upload fails immediately with a clear 422 — never a silently degraded index.
2. **Image handling for all three Mistral cases.** Inline `image_base64` is decoded and described (figures become searchable text); an existing `image_annotation` is spliced in as the description; an image with neither is replaced by an inert positional marker and **counted in `content_metadata`** — content loss is surfaced, never silent.
3. **Table splicing.** Side-channel `tables[].content` is spliced at its ref, whether Mistral returned HTML or markdown tables. Rows and columns survive into the chunk layer as HTML, so embeddings never see half a table.
4. **Noise removal.** Running headers and footers that repeat on every page are stripped — the same pass POMA's native pipeline applies — so your index isn't 4% page furniture.
5. **Heading enrichment.** Where Mistral's markdown under-marks headings (common in scanned or design-heavy documents), POMA's heading-injection pass restores them before chunking — the same treatment every natively ingested document gets.
6. **Hierarchy → chunks + chunksets.** The cross-page heading tree is rebuilt, and every sentence is emitted as a chunk with its `depth`, `page`, and a `to_embed` normalization — grouped into [chunksets](/learn/chunking/chunksets): root-to-leaf paths that keep each retrieved passage self-explanatory.

Because you already ran the OCR, **POMA charges only for the downstream structure and chunking value** — the OCR front-end is skipped entirely.

The result, measured on our reference legal-document benchmark: the same query answered with **337 tokens** of retrieved context instead of **1,542** for a recursive character splitter, with zero information loss. Methodology: [Document Ingestion & Chunking for RAG](/document-ingestion-chunking-rag).

## How to chunk Mistral OCR output with POMA

Run your OCR exactly as you do today, save the raw response, and hand the JSON to PrimeCut:

```python
import os
from mistralai import Mistral
from poma import PrimeCut

# 1. Your existing Mistral OCR call — unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
    model="mistral-ocr-latest",
    document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
    include_image_base64=True,  # lets POMA describe figures instead of dropping them
)
with open("contract.mistral-ocr.json", "w") as f:
    f.write(ocr.model_dump_json())

# 2. Chunk the raw result — POMA auto-detects the Mistral shape.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.mistral-ocr.json")

print(f"chunks: {len(result.chunks)}")
print(f"chunksets: {len(result.chunksets)}")
print(result.chunksets[0].to_embed)  # embedding-ready, hierarchy included
```

Two things worth knowing:

- **Auto-detection.** A JSON upload whose shape structurally fingerprints as a Mistral `/v1/ocr` result (`pages[]` carrying `markdown` + `index`) is routed through the BYOCR seam automatically. To force the declaration — or to opt a look-alike JSON *out* of detection — set the `external_ocr_source` parameter to `"mistral"` or `"none"` respectively on the ingest request.
- **Dedicated endpoint.** For explicit routing, the API exposes a `chunk_external_ocr_result` endpoint that accepts `external_ocr_source: "mistral"` (it supersedes the earlier Mistral-only `chunk_mistral_ocr_result` endpoint). See the [API reference](https://api.poma-ai.com/v3/docs).

From `result.chunks` / `result.chunksets` onward, everything is your stack: your vector store, your embedding model, your retrieval strategy. See the [Qdrant](/sdk/integrations/qdrant), [LangChain](/sdk/integrations/langchain), and [LlamaIndex](/sdk/integrations/llamaindex) integrations.

## Chunking options for Mistral OCR output, compared

| Approach | Structure used | Cross-page hierarchy | Tables | Images | Page numbers kept |
| --- | --- | --- | --- | --- | --- |
| Recursive character splitter | None | ✗ | Cut mid-row | Stripped or inlined as noise | ✗ |
| Markdown-header splitter | Per-page headings | ✗ | Usually survive | Usually stripped | Manual |
| Semantic chunker | Embedding similarity | ✗ | Fragile | Stripped | Manual |
| **POMA BYOCR (PrimeCut)** | **Full heading tree, rebuilt across pages** | **✓ chunksets** | **Spliced as HTML, never cut** | **Described or visibly counted** | **✓ per chunk** |

## Frequently asked questions

### How do I chunk Mistral OCR output for RAG?

Don't flatten the `/v1/ocr` response into one string and run a character splitter — you'd discard the page boundaries and heading structure Mistral already recovered. Feed the raw result JSON to a structure-aware chunker instead. POMA's BYOCR connector accepts the unmodified response, rebuilds the heading hierarchy across pages, and emits chunks plus chunksets ready for embedding.

### Can I use Mistral OCR results with POMA without re-running OCR?

Yes — that's the point of the bring-your-own-OCR path. POMA skips its OCR front-end and runs only the downstream structure, chunking, and retrieval-preparation stages on your result. Upload the raw JSON (auto-detected) or declare `external_ocr_source: "mistral"`.

### What does Mistral OCR return, exactly?

A `pages` array where each page carries an `index` and a `markdown` string, plus optional `images` (base64 when `include_image_base64` is set) and table side-channels. Structure is per-page; nothing links headings across pages — that's the chunker's job.

### Does a recursive character splitter work on Mistral OCR markdown?

It runs, but it ignores heading levels, cuts tables mid-row, and orphans section bodies from their headers. Markdown-header splitters do better but still emit fragments with no ancestor context.

### What happens to images in a Mistral OCR result during chunking?

POMA covers all three response cases: base64 bytes → decoded and described; existing annotation → spliced in; neither → inert positional marker, counted in `content_metadata`. Image loss is always visible.

### Should I switch OCR engines to fix retrieval quality?

Usually not. If Mistral OCR fits your cost, latency, or data-locality constraints, keep it — retrieval quality is mostly decided by chunking, not by swapping one strong OCR engine for another.

## Ship the chunks to your vector database

End-to-end recipes for wiring Mistral OCR into the store you already run:

[Qdrant](/pipelines/mistral-ocr-to-qdrant) · [Pinecone](/pipelines/mistral-ocr-to-pinecone) · [Weaviate](/pipelines/mistral-ocr-to-weaviate) · [pgvector](/pipelines/mistral-ocr-to-pgvector) · [Milvus](/pipelines/mistral-ocr-to-milvus) · [Chroma](/pipelines/mistral-ocr-to-chroma) — or browse [all pipeline recipes](/pipelines/).

## The rest of the series

The optimal chunker, for every OCR and parsing tool you already run:

- [LlamaParse](/optimal-chunker-llamaparse)
- [Azure Document Intelligence](/optimal-chunker-azure-document-intelligence)
- [Unstructured.io](/optimal-chunker-unstructured)
- [Docling](/optimal-chunker-docling)
- [Marker](/optimal-chunker-marker)
- [AWS Textract](/optimal-chunker-textract)

Or start with the fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [Document ingestion guide](/guides/document-ingestion/) · [POMA chunksets](/learn/chunking/chunksets).