Source: http://www.poma-ai.com/docs/optimal-chunker-deepseek-ocr

# DeepSeek-OCR Chunking for RAG: From Markdown to Retrieval Units

<ByAuthor />

DeepSeek-OCR is a 3B open-weights vision-language model that reads a page image and writes markdown. It runs on your own GPU, and it is unusually frugal with tokens. What it does not do is decide what a retrieval unit should be. That is the gap most searches for DeepSeek-OCR chunking are really about: the model hands you markdown, and something after it has to turn that markdown into chunks that still know which chapter they came from.

This page covers both halves. First what DeepSeek-OCR emits and how people run it. Then how to chunk that markdown structure-first, including the per-page assembly step, with an end-to-end pipeline into a vector store.

One thing up front, because it saves some readers the trouble: PrimeCut ingests a PDF directly and does the OCR itself. You do not need a separate OCR front-end at all. DeepSeek-OCR earns its place when you specifically want open weights reading your documents inside your own infrastructure.

## Quick start: DeepSeek-OCR markdown to chunksets

DeepSeek-OCR writes markdown. Markdown is a supported PrimeCut input type. So the join is one call: run the model as you already do, save the markdown, hand the file over.

```bash
pip install poma
```

```python
from poma import PrimeCut

# You already have contract.md from DeepSeek-OCR.
client = PrimeCut()                       # reads POMA_API_KEY
result = client.ingest("contract.md")     # markdown in, chunks + chunksets out

print(len(result.chunks), len(result.chunksets))
print(result.chunksets[0].to_embed)       # embedding-ready text with its heading path
```

PrimeCut rebuilds the heading hierarchy from the markdown, keeps tables whole, and returns chunks plus [chunksets](/learn/chunking/chunksets): root-to-leaf paths through the document tree, so every sentence carries its chapter, section and paragraph context into the embedding.

Two honest caveats about this path, spelled out rather than buried. DeepSeek-OCR is not one of the parser result formats PrimeCut auto-detects from its native JSON, so there is no `external_ocr_source` value for it. The route is the markdown file, nothing more. And because DeepSeek-OCR is a per-image model, page numbers come from the order in which you assemble the pages, not from the OCR. The [assembly helper](#assembling-per-page-markdown-before-chunking) below is where that order is fixed.

## What DeepSeek-OCR emits, and how people run it

The model is `deepseek-ai/DeepSeek-OCR` on Hugging Face: 3B parameters, BF16. Input is one image. Output is text, and with the document prompt that text is markdown.

The prompt selects the task. The README lists these verbatim:

```python
# document: <image>\n<|grounding|>Convert the document to markdown.
# other image: <image>\n<|grounding|>OCR this image.
# without layouts: <image>\nFree OCR.
# figures in document: <image>\nParse the figure.
# general: <image>\nDescribe this image in detail.
# rec: <image>\nLocate <|ref|>xxxx<|/ref|> in the image.
```

`<|grounding|>` asks the model to tie what it reads to where it sits on the page, so the output carries layout reference markers alongside the text. `Free OCR.` returns text without them. For a retrieval pipeline, either works, as long as the file you feed downstream is clean markdown: strip the grounding markers, or use the non-grounding prompt.

Resolution is the other dial, and it is the interesting one. The open-source model supports five modes:

| Mode | Resolution | Vision tokens |
| --- | --- | --- |
| Tiny | 512×512 | 64 |
| Small | 640×640 | 100 |
| Base | 1024×1024 | 256 |
| Large | 1280×1280 | 400 |
| Gundam | dynamic, n×640×640 + 1×1024×1024 | n tiles plus one global view |

In code the mode is three arguments. The model card maps them: Tiny is `base_size=512, image_size=512, crop_mode=False`; Small `640/640/False`; Base `1024/1024/False`; Large `1280/1280/False`; Gundam `base_size=1024, image_size=640, crop_mode=True`.

**Transformers.** The README's example, with Gundam settings:

```python
from transformers import AutoModel, AutoTokenizer
import torch

model_name = "deepseek-ai/DeepSeek-OCR"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_name,
    _attn_implementation="flash_attention_2",
    trust_remote_code=True,
    use_safetensors=True,
)
model = model.eval().cuda().to(torch.bfloat16)

prompt = "<image>\n<|grounding|>Convert the document to markdown. "
res = model.infer(
    tokenizer,
    prompt=prompt,
    image_file="page_001.png",
    output_path="out/",
    base_size=1024,
    image_size=640,
    crop_mode=True,
    save_results=True,
    test_compress=True,
)
```

The pinned environment on the model card is `torch==2.6.0`, `transformers==4.46.3`, `tokenizers==0.20.3`, plus `flash-attn==2.7.3`.

**vLLM.** DeepSeek-OCR has been supported in upstream vLLM since 23 October 2025. The recipe loads the model with `enable_prefix_caching=False`, `mm_processor_cache_gb=0`, and `logits_processors=[NGramPerReqLogitsProcessor]` from `vllm.model_executor.models.deepseek_ocr`, then batches `{"prompt": ..., "multi_modal_data": {"image": ...}}` entries. Sampling uses `temperature=0.0`, `max_tokens=8192`, and `extra_args=dict(ngram_size=30, window_size=90, whitelist_token_ids={128821, 128822})`, which whitelists the `<td>` and `</td>` tokens so the repetition guard does not eat table markup. This is the throughput path; the repository quotes roughly 2500 tokens/s on a single A100-40G for its PDF script.

**Ollama.** For a laptop, `ollama pull deepseek-ocr` then `ollama run deepseek-ocr "/path/to/image\n<|grounding|>Convert the document to markdown."`. It needs Ollama v0.13.0 or newer. The registry listing shows a 6.7 GB download and an 8K context window. Ollama's own note is worth repeating: the model is sensitive to its input, and a missing newline or piece of punctuation in the prompt can produce a bad result.

## Contexts optical compression, explained accurately

The paper behind the model is [DeepSeek-OCR: Contexts Optical Compression](https://arxiv.org/abs/2510.18234) (arXiv:2510.18234). The idea is not a chunking strategy, and reading it as one leads people astray.

The premise: a page of text costs a lot of text tokens, but a picture of that page costs only vision tokens, and there are far fewer of them. So render the context as an image and let a vision encoder carry it. The paper's headline result is about how far that trade goes before it breaks. When the text token count is within ten times the vision token count, so a compression ratio under 10x, decoding precision stays around 97%. At 20x compression, accuracy falls to roughly 60%. The efficiency comparisons follow from the same design: around 100 vision tokens where GOT-OCR2.0 uses 256, and under 800 where MinerU2.0 spends over 6000 per page.

This is a real and useful result. It is also about a different problem than retrieval. Optical compression is about fitting more document into an LLM's context window. Chunking is about deciding which slice of a document should be one embedding vector, and how much surrounding context that slice must carry to still make sense on its own when it comes back from a search. Compressing the input to a model does nothing for either question. A 64-token Tiny-mode page and a 400-token Large-mode page produce the same downstream problem: markdown that has to be cut somewhere.

DeepSeek-OCR2 arrived in January 2026 as a separate repository; the details on this page describe the original model.

## Where DeepSeek-OCR stops

Three limits matter for a retrieval pipeline, and none of them is a criticism of the model. They are simply outside its job.

- **No chunking.** The output is display-ready markdown, not retrieval units. Nothing in the model has an opinion about what belongs in one vector.
- **Per image, therefore per page.** You call it with one image at a time. A 300-page PDF is 300 calls and 300 outputs. Whatever concatenates them decides the document order.
- **No cross-page hierarchy.** A `## Termination clauses` heading recovered on page 41 has no machine-readable link back to the `# Master Services Agreement` chapter that opened on page 3. The heading levels are correct within a page. The tree across pages is not built.

The third one is what quietly costs retrieval quality. A chunk that reads "either party may terminate on 30 days notice" is useless when the index cannot say which agreement it belongs to. That failure mode is covered in depth in the [chunking strategies guide](/rag-chunking-strategies-text-splitters).

## Assembling per-page markdown before chunking

If you ran the model per page, concatenate in page order first, then ingest the single file. Sorting matters: `sorted()` on `page_2.md` and `page_10.md` puts 10 before 2, so pad the filenames or sort numerically.

```python
from pathlib import Path

pages = sorted(Path("out").glob("page_*.md"), key=lambda p: int(p.stem.split("_")[1]))
Path("contract.md").write_text("\n\n".join(p.read_text() for p in pages))
```

Then `PrimeCut().ingest("contract.md")` as in the quick start. Be clear about what this means for citations: the page numbers on the resulting chunks come from the order you just assembled, not from anything DeepSeek-OCR reported. Get the sort wrong and the page numbers are wrong, silently. If page-accurate citation is a hard requirement, hand PrimeCut the original PDF instead and let it own both the OCR and the pagination.

## Chunking DeepSeek-OCR markdown for RAG

The default move after OCR is to load the markdown and run `RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)` over it. That throws away most of what the model just recovered:

| What DeepSeek-OCR recovered | What a character splitter does with it |
| --- | --- |
| Heading levels (`#`, `##`, `###`) | Ignored; cuts land wherever the character count falls |
| Tables | Sliced mid-row, header row separated from its data |
| Reading order across columns | Preserved only by luck |
| Page boundaries | Already gone at concatenation |
| Figure descriptions from `Parse the figure.` | Stripped or fragmented |

A markdown-header splitter is better: cuts land on structural boundaries. It still emits isolated fragments. The passage under `### Early termination` no longer knows it sits inside `## Termination clauses` inside `# Master Services Agreement`, so at answer time the model reads an orphan.

PrimeCut takes the other route. It rebuilds the heading tree across the whole assembled document, keeps tables intact through the chunk layer, and returns each sentence as a chunk record with its `depth` (and `page` in the archive's chunk records) and an embedding-ready `to_embed` on the chunkset, grouped into chunksets that keep every retrieved passage self-explanatory. No overlap is needed, because context comes from the hierarchy rather than from duplicated boundary text.

One illustration of what that is worth: on a legal document in our reference set, the same question was answered from 337 tokens of retrieved context with chunksets versus 1,542 with a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the [ingestion guide](/document-ingestion-chunking-rag) has the broader numbers.

## The DeepSeek-OCR RAG pipeline, end to end

Assemble the pages, chunk, embed and store, then retrieve. Qdrant here; the same shape applies to any store.

```bash
pip install 'poma[qdrant]'
```

```python
import os
from pathlib import Path
from poma import PrimeCut
from poma.integrations.qdrant import PomaQdrant

# 1. Your DeepSeek-OCR run already wrote out/page_0001.md … out/page_0300.md.
pages = sorted(Path("out").glob("page_*.md"), key=lambda p: int(p.stem.split("_")[1]))
Path("contract.md").write_text("\n\n".join(p.read_text() for p in pages))

# 2. Chunk: markdown in, chunks + chunksets out.
poma = PrimeCut()
result = poma.ingest("contract.md")

# 3. Qdrant: hybrid dense + sparse points with hierarchy payloads, one call.
qdrant = PomaQdrant(
    url=os.environ["QDRANT_URL"],
    api_key=os.environ["QDRANT_API_KEY"],
    cloud_inference=False,  # True needs Qdrant Cloud inference; False embeds locally via fastembed
    collection_name="contracts",
    dense_model="sentence-transformers/all-MiniLM-L6-v2",
    sparse_model="Qdrant/bm25",
    dense_size=384,
    auto_create_collection=True,
)
qdrant.upsert_poma_points(result)

# 4. Retrieve prompt-ready context: hits merged along their shared heading path.
cheatsheets = qdrant.get_cheatsheets(
    query="What are the early termination conditions?",
    limit=3,
)
print(cheatsheets[0]["content"])
```

With `cloud_inference=False` and a self-hosted Qdrant, the only thing that leaves your infrastructure is the assembled markdown going to POMA for chunking. The model weights, the source PDFs and the vector index all stay put.

## DeepSeek-OCR compared with Mistral OCR and PaddleOCR-VL

A short, honest comparison, since these three come up together.

DeepSeek-OCR is open weights you run yourself, cheapest in vision tokens, and per image. It gives you markdown and nothing structured around it. [Mistral OCR](/optimal-chunker-mistral-ocr) is a hosted API that returns a JSON envelope with one entry per page: page indexes, tables in a side-channel, images, and typed blocks with bounding boxes. That envelope carries more machine-readable structure than a markdown file does, which is why PrimeCut consumes it natively. [PaddleOCR-VL](/optimal-chunker-paddleocr-vl) sits in between: self-hosted like DeepSeek-OCR, but with an explicit layout stage that emits labelled blocks and a reading-order field, and also natively supported.

All three are strong readers. Pick on the constraints that actually differ: hosted versus self-hosted, cost, data locality, and whether you want structured JSON or plain markdown coming out. None of them chunks for retrieval, so the piece after them is the same either way.

## Frequently asked questions

### How do I chunk DeepSeek-OCR output for RAG?

Save the markdown the model writes, concatenate the per-page files in page order if you ran it per page, and hand the single file to a structure-aware chunker. PrimeCut accepts markdown as an input type, rebuilds the heading hierarchy across the whole document, keeps tables whole, and returns chunks plus chunksets ready to embed. Do not run a character splitter over the markdown: it cuts wherever the character count lands and discards the heading structure DeepSeek-OCR just recovered.

### Does DeepSeek-OCR chunk documents itself?

No. DeepSeek-OCR converts one page image into markdown. It has no notion of a retrieval unit, no cross-page heading tree, and no chunking parameters. Deciding what one embedding vector should represent is left entirely to the pipeline after it.

### What is DeepSeek-OCR context compression, and does it help RAG?

It is the idea from the paper DeepSeek-OCR: Contexts Optical Compression, arXiv:2510.18234: render text as an image so a vision encoder can carry it in far fewer tokens. The paper reports about 97% decoding precision at compression ratios under 10x, falling to roughly 60% at 20x. That is about fitting more document into an LLM context window. Retrieval chunking is a separate problem, and optical compression does not address it. The markdown still has to be cut into units somewhere.

### How do I run DeepSeek-OCR: vLLM, transformers, or Ollama?

All three work. Transformers is the reference path, using model.infer with base_size, image_size and crop_mode to pick the resolution mode. vLLM has upstream support since October 2025 and is the throughput path, loading the model with NGramPerReqLogitsProcessor and whitelisting the table tokens so the repetition guard does not damage table markup. Ollama is the easy local path: ollama run deepseek-ocr, requiring Ollama v0.13.0 or newer, at a 6.7 GB download.

### Which DeepSeek-OCR resolution mode should I use for documents?

The open-source model offers Tiny at 512x512 with 64 vision tokens, Small at 640x640 with 100, Base at 1024x1024 with 256, Large at 1280x1280 with 400, and dynamic Gundam mode combining n tiles of 640x640 with one 1024x1024 global view. Dense multi-column pages and small print need the higher modes or Gundam. The choice affects OCR fidelity, not chunking: every mode produces markdown that still has to be turned into retrieval units.

### Does PrimeCut auto-detect DeepSeek-OCR output?

No. PrimeCut auto-detects the native result formats of a specific set of parsers, and DeepSeek-OCR is not among them. The supported route is its markdown output, ingested as a .md file. That works well, because markdown headings are exactly what the hierarchy rebuild needs.

### How do page numbers work when chunking DeepSeek-OCR markdown?

DeepSeek-OCR runs per image, so it reports no document-level page index. If you concatenate per-page markdown files, the page numbers on the resulting chunks follow the order in which you assembled them. Sort the filenames numerically, not lexically, or page 10 lands before page 2. If page-accurate citation is critical, give PrimeCut the original PDF and let it handle pagination itself.

### Do I need DeepSeek-OCR at all if PrimeCut parses PDFs?

No. PrimeCut does ingestion and chunking, so a PDF can go straight in and OCR happens inside. DeepSeek-OCR is worth adding when you want an open-weights reader you control: documents that cannot leave your infrastructure at the OCR stage, or a per-page cost you want to own on your own hardware.

## Continue reading

- [RAG chunking strategies and text splitters](/rag-chunking-strategies-text-splitters)
- [POMA chunksets](/learn/chunking/chunksets)
- [AWS Textract](/optimal-chunker-textract)
- [Azure Document Intelligence](/optimal-chunker-azure-document-intelligence)
- [Docling](/optimal-chunker-docling)
- [LlamaParse](/optimal-chunker-llamaparse)
- [Marker](/optimal-chunker-marker)
- [MinerU](/optimal-chunker-mineru)
- [Mistral OCR](/optimal-chunker-mistral-ocr)
- [PaddleOCR-VL](/optimal-chunker-paddleocr-vl)
- [Unstructured.io](/optimal-chunker-unstructured)
- [All pipeline recipes](/pipelines/)