Source: http://www.poma-ai.com/docs/guides/ocr-for-rag/

# OCR for RAG: Choosing an Engine and Chunking Its Output

<ByAuthor />

Choosing OCR for RAG is two decisions, not one. First, which engine turns your documents into text. Second, what happens to that text before it becomes embedding vectors. Most write-ups only cover the first, which is why so many pipelines with a good parser still retrieve badly.

This page covers both. It states what an OCR engine has to preserve for retrieval to work, compares ten engines by fit rather than by rank, and shows where in the pipeline quality is actually lost. If you have no engine yet, PrimeCut does the parsing and the chunking in one call and you can skip the comparison. If you already run Mistral, Docling, Textract or another engine, keep it — the section on chunking its output is the part you need.

<Tldr>
Pick an OCR engine by hosting model, output format and document type, not by a text-accuracy leaderboard. Once the engine parses your documents cleanly, further OCR upgrades buy little; the remaining loss is in chunking. Keep the engine's native result, chunk it with the structure intact, and carry page numbers into the vector store.
</Tldr>

## What OCR has to deliver for RAG

A PDF encodes glyph positions, not meaning. An OCR engine's job in a retrieval pipeline is to recover the meaning a human reader infers from the layout. Five things matter, in roughly this order.

**Reading order.** Two-column layouts, sidebars, footnotes and rotated tables all break naive left-to-right extraction. A parse that interleaves two columns produces sentences that never existed. Nothing downstream can repair that.

**Heading levels.** Headings are the document's own outline, and they are the single most useful chunking signal available. An engine that emits `## Termination clauses` gives you a boundary; one that emits the same string as bold body text does not.

**Whole tables.** A table sliced mid-row is worse than no table: "Revenue 2023 4.2 2024 5.1" is confidently wrong context. Look for an engine that emits table structure separately, as HTML or as cell grids, so a chunker can keep it intact.

**Page numbers.** If a chunk cannot say which page it came from, your answers cannot cite. Page provenance has to survive from the OCR result all the way into the vector store payload.

**Figures.** Charts and diagrams carry content that exists nowhere in the text. The choice is to describe them, or at minimum to mark their position so the loss is visible rather than silent.

Notice what is not on that list: character accuracy. It is the number every engine benchmarks and the wrong metric to optimise alone. A 99.5% accurate parse that interleaves two columns is useless for retrieval; a 98% accurate parse with correct reading order and intact tables works fine, because embeddings tolerate typos and do not tolerate scrambled context.

## OCR quality is a ceiling, not a knob

Accuracy still sets an upper bound. mixedbread's [analysis of OCR quality](https://www.mixedbread.com/blog/the-hidden-ceiling) benchmarked several extraction methods against ground-truth text over thousands of enterprise PDF pages and found that extraction errors cap both stages of RAG at once: corrupted text makes relevant passages unfindable at retrieval, and then feeds the model flawed context at generation. The best OCR methods still trailed perfect text on both retrieval scores and answer correctness, with the gap widest on tables, charts and handwriting.

The practical reading is a ceiling, not a dial. Below a threshold, no amount of downstream engineering recovers a bad parse. Above it, the engines converge, and the difference between two competent parsers is much smaller than the difference between chunking their output well and chunking it badly. Test three engines on twenty of your own real documents, pick the one that handles your worst layouts, and then stop tuning OCR.

## Choosing OCR for RAG: ten engines compared

Hosted engines cost per page and require sending documents out. Self-hosted engines cost GPU time and operational effort, and keep data in your infrastructure. That axis usually decides more than accuracy does.

| Engine | Hosting | Output | Structure recovered | Tables | Cost model | Best for |
| --- | --- | --- | --- | --- | --- | --- |
| [Mistral OCR](/optimal-chunker-mistral-ocr) | Hosted API; self-deployable container | Markdown per page, JSON envelope | Headings, page index, typed blocks with boxes and confidence scores (OCR 4) | Inline or `tables[]` side-channel | Priced per 1,000 pages; the Batch API halves it | Broad multilingual document sets with page-level citation |
| [LlamaParse](/optimal-chunker-llamaparse) | Hosted API | Markdown, text or JSON | Layout-aware, per-page `md`, 130+ file types | Cell structure preserved | Credits per page, four tiers from Fast to Agentic Plus | Visually complex documents where a harder tier is worth paying for |
| [Azure AI Document Intelligence](/optimal-chunker-azure-document-intelligence) | Azure cloud or container | JSON, or markdown via `output_content_format` | Paragraphs, roles, sections, page spans from `prebuilt-layout` | Structured cell grids | Azure per-page pricing | Teams already on Azure with compliance requirements |
| [Amazon Textract](/optimal-chunker-textract) | AWS cloud | `Blocks[]` JSON | `LAYOUT` blocks including `LAYOUT_TITLE` and `LAYOUT_SECTION_HEADER`; forms, queries, signatures | `TABLE` blocks with cells | AWS per-page, per-feature | AWS-native pipelines and forms-heavy documents |
| [Unstructured](/optimal-chunker-unstructured) | Hosted API and open-source library | Typed element list as JSON | Title, NarrativeText, Table, Image elements with metadata | `metadata.text_as_html` with `infer_table_structure` | Per page on the API; library is free | Mixed corpora across many file types |
| [Docling](/optimal-chunker-docling) | Self-hosted (MIT) | DoclingDocument, plus markdown, HTML, JSON | Reading order, section header levels, code, formulas, chart understanding | Table structure model | Your own compute | The broadest local option, air-gapped environments |
| [MinerU](/optimal-chunker-mineru) | Self-hosted (MinerU Open Source License, Apache-2.0 based) | Markdown and reading-order JSON | Reading order, footnotes, furniture removal, cross-page table merging | HTML tables, LaTeX formulas | Your own compute; CPU pipeline or GPU VLM backend | Scientific and technical PDFs with heavy maths |
| [PaddleOCR-VL](/optimal-chunker-paddleocr-vl) | Self-hosted (Apache 2.0) | Markdown and JSON | Layout labels including `doc_title` and `paragraph_title`, coordinates | Table and cell recognition | Your own compute; ~0.9B-parameter model | Very wide language coverage on modest hardware |
| [Marker](/optimal-chunker-marker) | Self-hosted; hosted option from Datalab | Markdown, JSON block tree, HTML, chunks | Reading order, headers and footers, equations as LaTeX, forms | Full HTML tables, page merging with `--use_llm` | Your own compute; code Apache-2.0, weights under a modified AI Pubs Open RAIL-M license | Fast local conversion when the weight license fits |
| [DeepSeek-OCR](/optimal-chunker-deepseek-ocr) | Self-hosted (MIT) | Markdown | Document-to-markdown conversion with grounding | Emitted in markdown | Your own compute; CUDA, vLLM or Transformers | Permissive licensing and research on optical context compression |

Two notes on the self-hosted column. Marker's code is Apache-2.0 but its model weights are not, and the license is free only below a stated revenue and funding threshold, so read it before shipping. DeepSeek-OCR is a research release exploring how few vision tokens a page can be compressed into, from 64 at 512x512 up to 400 at 1280x1280; it is genuinely interesting and genuinely not a managed service.

Everything in the table is a competent parser. None of them chunks for retrieval, and that is the next problem.

## The pipeline after OCR, and where quality is lost

The full path is: OCR, chunk, embed, store, retrieve. Each stage can only work with what the previous one passed on.

**Chunking** is where most pipelines throw the parse away. The default move is to concatenate the engine's markdown into one string and run a recursive character splitter over it. That discards page boundaries at the moment of concatenation, cuts tables mid-row, and separates headings from the sections they introduce. You paid an engine to recover structure and then deleted it in one line.

Header-aware splitters are better: cuts land on real boundaries. They still produce isolated fragments. A chunk under `### Early termination` no longer knows it sits inside `## Termination clauses` inside `# Master Services Agreement`, because heading hierarchy in most engines is per page and flat. At retrieval time the model reads an orphaned paragraph. The [chunking strategies guide](/rag-chunking-strategies-text-splitters) covers the full taxonomy of approaches.

**Embedding** is the stage most teams tune first and it is rarely the bottleneck. An embedding model can only encode what the chunk contains. If the chunk is a fragment, a better model produces a better vector for a fragment.

**Storage** is where provenance is kept or dropped. Carry `file_id` and `chunkset_index`, plus the leaf chunk's `depth` and `page` from the archive's chunk records, into the payload alongside the vector. Without page, no citation; without depth, no way to widen a hit to its parent section.

**Retrieval** is bounded by the index. Hybrid search and reranking both help, and neither can return context that was never stored. If your retrieved passages are incoherent when you read them, the fix is upstream of the retriever.

The short version: once OCR is decent, chunking is the bottleneck. That is the opposite of where most engine-selection effort goes.

## Where PrimeCut fits

PrimeCut is an ingestion and chunking engine. It covers both roles above, and you can use either half.

**No engine yet.** Hand it the file. PDF, images, HTML, markdown, txt, json, csv and xlsx are all supported upload types; parsing and chunking happen in one call. This is the default path and the shortest pipeline.

```python
from poma import PrimeCut

client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.pdf")

print(len(result.chunks), len(result.chunksets))
print(result.chunksets[0].to_embed)  # embedding-ready text with its heading path
```

**Engine already in production.** Keep it. Save its native result and upload that instead of a PDF. Results from Mistral, LlamaParse, Azure Document Intelligence, Unstructured, Docling, Marker, Textract and PaddleOCR-VL are auto-detected by shape, so the same `ingest` call runs chunking only and skips PrimeCut's own OCR front-end. Set `external_ocr_source` on the request to force or suppress detection.

```python
# Mistral OCR result JSON, saved unmodified. Same call, OCR skipped.
result = client.ingest("contract.mistral-ocr.json")
```

For engines outside that list, including MinerU, DeepSeek-OCR and PyMuPDF, there is no JSON path. Their markdown output can be ingested as a `.md` file, because markdown is a supported upload type. That is the whole story for those engines: no shape detection, no side-channel tables, just their markdown.

What comes back either way is chunks and [chunksets](/learn/chunking/chunksets). A chunkset is a root-to-leaf path through the document hierarchy, so every sentence arrives with its chapter, section and paragraph context attached. Tables stay whole, page numbers stay on each chunk, and no overlap is needed because the units nest instead of sliding. On one legal document in our reference set, a question was answered from 337 tokens of retrieved context with chunksets against 1,542 with a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark.

Downstream, the store is your choice. The SDK ships integrations for LangChain, LlamaIndex and Qdrant; for anything else, embed `chunkset.to_embed` with your own model and upsert with the metadata fields above.

## How to choose, in four questions

1. **Can the documents leave your network?** No: Docling, MinerU, PaddleOCR-VL, Marker, DeepSeek-OCR, or a self-deployed Mistral OCR container. Yes: the hosted APIs are less work.
2. **Which cloud are you already in?** Azure Document Intelligence and Textract are the paths of least resistance inside Azure and AWS respectively, and the billing is already set up.
3. **What is your hardest document?** Dense maths favours MinerU. Forms favour Textract. Very wide language coverage favours PaddleOCR-VL or Mistral OCR. Test on your own worst pages, not on a benchmark corpus.
4. **Do you have an engine at all?** If not, use PrimeCut's own ingestion and skip the selection problem until you have evidence you need a specialist.

## Frequently asked questions

### What is the best OCR for RAG?

There is no single best OCR for RAG. Pick by constraint. If you want a hosted API with markdown output and per-page pricing, Mistral OCR, LlamaParse, Azure AI Document Intelligence, Amazon Textract and Unstructured all qualify, and your cloud and compliance situation usually decides between them. If the documents cannot leave your infrastructure, Docling, MinerU, PaddleOCR-VL, Marker and DeepSeek-OCR all run locally. What matters more than the ranking is that the engine preserves reading order, heading levels, whole tables and page numbers, and that you chunk its output with that structure intact instead of flattening it to a string.

### How do I build an OCR RAG pipeline?

Five stages: OCR the document, chunk the OCR output, embed the chunks, store them with metadata, and retrieve. The two mistakes that cost the most quality are flattening the OCR result to plain text before chunking, and splitting that text by character count. Keep the engine's native result, chunk it with a structure-aware chunker that rebuilds the heading hierarchy across pages, and carry file id, page number, depth and chunk index into the vector store so answers can cite a page.

### What is the best open source OCR for RAG?

Docling (MIT) is the broadest: many input formats, a typed DoclingDocument tree, local execution and OCR for scans. MinerU is strong on scientific PDFs, converts formulas to LaTeX and tables to HTML, and offers a CPU-capable pipeline backend alongside a GPU VLM backend. PaddleOCR-VL is a 0.9B-parameter Apache-2.0 model covering a wide language range with markdown and JSON output. Marker is Apache-2.0 in code but its model weights use a modified AI Pubs Open RAIL-M license, so check the revenue threshold before commercial use. DeepSeek-OCR is MIT and the most permissive, but it is a research model you operate yourself.

### Is a vision language model better than OCR for RAG?

The distinction has mostly collapsed. Mistral OCR, PaddleOCR-VL, DeepSeek-OCR, MinerU's VLM backend and Unstructured's vlm strategy are all vision language models applied to document parsing, and they return text, not embeddings. A separate approach, embedding page images directly with a multimodal retriever, skips text entirely, but it gives up page-level citation into text, exact keyword matching and cheap re-indexing. For document RAG, running a VLM-based parser and then chunking its text output is the practical default.

### Does OCR quality limit RAG accuracy?

Yes, up to a point. mixedbread's analysis of OCR quality across thousands of enterprise PDF pages found that extraction errors cap both retrieval and generation, with the best OCR methods still trailing ground-truth text on retrieval scores and answer correctness. The effect is largest on tables, charts and handwriting. The practical reading is that a bad parse cannot be recovered downstream, but once your engine parses your documents cleanly, further OCR upgrades buy little and the remaining loss is in chunking.

### How do I use Mistral OCR for RAG?

Call /v1/ocr, save the raw JSON envelope rather than a concatenation of pages[].markdown, and chunk that JSON. The envelope carries page indexes, tables, images and, since OCR 4, typed blocks with bounding boxes and confidence scores, all of which a character splitter discards. PrimeCut auto-detects the Mistral shape, so client.ingest on the saved JSON runs chunking only and skips its own OCR front-end.

### Can I use DeepSeek-OCR for RAG?

Yes. DeepSeek-OCR is MIT-licensed and converts document images to markdown, which is a supported PrimeCut upload type, so you can ingest its markdown output as a .md file and get chunks and chunksets from it. It is not in the auto-detected external OCR result formats, so there is no JSON path for it. Note that it is a research release you run yourself on CUDA, via vLLM or Transformers, not a hosted service.

### Do I still need a separate OCR engine with PrimeCut?

No. PrimeCut does ingestion and chunking: hand it a PDF, image, Office file, HTML or markdown and it parses and chunks in one call. The bring-your-own-OCR path exists for teams that already run an engine and want to keep it. If you have no engine yet, PrimeCut's own ingestion is the default and the shorter pipeline.

## Continue reading

- [Document Ingestion & Chunking for RAG](/document-ingestion-chunking-rag) — the parsing layer in depth, and how to build a production ingestion pipeline
- [The Ultimate Guide to RAG Chunking Strategies & Text Splitters](/rag-chunking-strategies-text-splitters) — every chunking approach compared, with tradeoffs
- [What is Context Engineering?](/guides/context-engineering/) — why context quality is decided before the prompt
- [POMA chunksets](/learn/chunking/chunksets) — how root-to-leaf retrieval units work
- [RAG architecture](/guides/rag-architecture/) — the full stack around the ingestion layer

<AuthorBio />