Source: http://www.poma-ai.com/docs/optimal-chunker-mineru

# MinerU Chunking for RAG: From Markdown Output to Chunksets

<ByAuthor />

MinerU does not chunk. It parses. Run `mineru -p contract.pdf -o out/` and you get layout-aware Markdown, a flat content list in reading order, and an images directory. What you do not get is retrieval units: nothing in that output says which sentences should travel together into one embedding vector. That decision is yours. MinerU chunking is a separate step from MinerU parsing, and it is where most MinerU RAG pipelines quietly lose quality.

This page covers the second half of the job: what MinerU writes, why running a character splitter over the Markdown throws away the structure MinerU just recovered, and how to chunk that Markdown into units that keep their heading path. It also explains what MinerU is doing inside RAGFlow and RAG-Anything, since that is how many teams meet it.

## Quick start: MinerU PDF to Markdown, then chunk

Two commands. MinerU parses, PrimeCut chunks the Markdown it wrote.

```bash
pip install poma
mineru -p contract.pdf -o out/
```

MinerU writes its results under `out/<name>/<backend>/` — `out/contract/pipeline/contract.md` for the pipeline backend, `out/contract/vlm/contract.md` for a VLM one. Globbing avoids hardcoding that:

```python
import glob
from poma import PrimeCut

md_path = glob.glob("out/contract/*/contract.md")[0]

client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest(md_path)  # markdown is a first-class upload type

print(len(result.chunks), len(result.chunksets))
print(result.chunksets[0].to_embed)  # embedding-ready text with its heading path
```

Markdown is one of PrimeCut's supported upload types, so no adapter, no conversion, no re-parsing. The `#`, `##` and `###` levels MinerU derived from `text_level` are read back as a hierarchy, tables stay whole, and every sentence comes out with the chapter and section it belongs to. From `result.chunks` and `result.chunksets` onward the stack is yours; the [end-to-end example](#end-to-end-mineru-to-qdrant) below wires it into Qdrant.

## What MinerU writes, and where it stops

One MinerU run produces several artefacts per document ([output file reference](https://opendatalab.github.io/MinerU/reference/output_files/)):

| File | Contents |
| --- | --- |
| `<name>.md` | The Markdown conversion: headings, paragraphs, lists, tables, formulas, image references |
| `<name>_content_list.json` | A flat list of every readable content block, in reading order |
| `<name>_content_list_v2.json` | The v3.0+ structured output, generated by every backend: blocks grouped by page under a unified `type` + `content` shape |
| `<name>_middle.json` | The hierarchical intermediate result: `pdf_info` per page, then blocks, lines and spans |
| `<name>_model.json` | Raw model inference output |
| `<name>_layout.pdf`, `<name>_span.pdf` | Visual debugging overlays |
| `images/` | Extracted figures and table crops referenced from the Markdown |

The content list is the interesting sidecar. Each entry carries:

- **`type`** — `text`, `table`, `image`, `chart`, `equation`, `code`, `list`, plus auxiliary types such as `header`, `footer` and `page_number`.
- **`text_level`** — heading depth. Absent or `0` means body text, `1` is a level-one heading, `2` a level-two heading, and so on. This is the field the `#` levels in the Markdown come from.
- **`page_idx`** — the source page, counting from 0.
- **`bbox`** — the block's box as `[x0, y0, x1, y1]`, normalised to a 0–1000 range.
- **`img_path`**, `image_caption`, `table_body` and other type-specific fields.

That is genuinely good structural recovery: reading order across columns, headings with an explicit level, tables as table bodies rather than smeared text, formulas kept as equations. The CLI exposes the knobs that matter — `-m` with `auto`, `txt` or `ocr`, `-b` with `pipeline`, `vlm-engine`, `hybrid-engine`, `vlm-http-client` or `hybrid-http-client`, plus `-l` for OCR language, `-f`/`-t` for formula and table parsing, and `-s`/`-e` for a page range ([CLI reference](https://opendatalab.github.io/MinerU/usage/cli_tools/)).

What MinerU deliberately leaves undone:

- **Retrieval units.** There is no chunk anywhere in the output. The Markdown is a document, the content list a block stream; neither is sized for an embedding model.
- **Cross-block lineage.** `text_level` says a block is a level-three heading. It does not say which level-one chapter the block underneath it belongs to. Reconstructing that ancestry is a separate step.
- **Figure semantics.** Images are extracted as files with references left behind. Nothing turns a chart into text a retriever can match.

## Why the content list is worth keeping even if you chunk the Markdown

The common instinct is to pick one artefact and delete the rest. Keep `<name>_content_list.json` anyway, for three reasons that have nothing to do with the chunker.

**Page numbers.** MinerU's Markdown has no page delimiters; `page_idx` lives only in the JSON sidecars. If you want an answer to cite page 41, the content list is the cheapest way back: match chunk text against block text and carry `page_idx` into your vector store payload.

**Auditing what was dropped.** When a table looks wrong in retrieval, `bbox` plus `_layout.pdf` tells you in seconds whether MinerU mis-detected the region or your chunker mangled a correct parse. Without the sidecar, every quality question turns into a re-run.

**Filtering furniture.** The `header`, `footer` and `page_number` types mark exactly the repeated furniture you do not want in embeddings — a useful cross-check on whatever your chunker strips.

One boundary to be precise about: `content_list.json` is **not** a shape PrimeCut auto-detects. Its bring-your-own-parser path recognises result JSON from a fixed set of engines, and MinerU is not among them. The supported route is the Markdown file, ingested as Markdown. A declared-only generic path exists (`external_ocr_source: "anydoc"`), but it accepts a plain UTF-8 Markdown upload rather than a block-list JSON, so it buys nothing over ingesting the `.md`. Treat the content list as your metadata source, not as an ingest input.

## MinerU inside RAGFlow and RAG-Anything

Most people who search for MinerU RAG advice are already running it indirectly.

**RAGFlow** added MinerU as an optional PDF parser in v0.21.1. RAGFlow acts as a remote client: you stand up a MinerU FastAPI service, point `MINERU_APISERVER` at it, and select MinerU from the PDF parser dropdown for a dataset or in a Parser component ([RAGFlow release notes](https://github.com/infiniflow/ragflow/blob/main/docs/release_notes.md)). RAGFlow then applies its own chunking method to the returned files.

**RAG-Anything** names MinerU as its default parser and selects it with `parser="mineru"`, alongside `parse_method` values of `auto`, `ocr` or `txt` that map onto MinerU's `-m` flag ([RAG-Anything README](https://github.com/HKUDS/RAG-Anything)). The parsed content then flows into its LightRAG-based indexing.

The division of labour is the same in both: MinerU supplies layout-aware structure, the framework supplies generic chunking. That is why swapping in a better parser often moves retrieval quality far less than teams expect. If your MinerU output looks great in the preview but your answers are still vague, the splitter is the suspect.

## Why generic text splitters waste MinerU Markdown

Concatenate the Markdown, run `RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)`, embed. Here is what that costs:

| What MinerU recovered | What a character splitter does with it |
| --- | --- |
| Heading levels from `text_level` | Ignored; cuts land wherever the character budget runs out |
| Tables (`table_body`, HTML or Markdown) | Sliced mid-row, header row orphaned from data rows |
| Formulas kept as equations | Broken across chunk boundaries into meaningless fragments |
| Reading order across multi-column layouts | Preserved only by luck of the cut positions |
| Figure references | Embedded as literal `![](images/xyz.jpg)` noise |
| `page_idx` from the content list | Never consulted; chunks cannot cite a page |

A Markdown-header splitter is a real improvement: cuts land on structural boundaries instead of character counts. It still yields **isolated fragments**. A chunk under `### Early termination` no longer knows it sits inside `## Termination clauses` inside `# Master Services Agreement`. The retriever surfaces an orphaned paragraph and the model answers without the context that made it meaningful. The [chunking strategies guide](/rag-chunking-strategies-text-splitters) has the full taxonomy.

## MinerU chunking in practice: Markdown into chunksets

PrimeCut is an ingestion and chunking engine: hand it a PDF and it parses too. The path on this page is the other one — you already run MinerU, so only the chunking stages execute on its Markdown:

1. **Hierarchy rebuild.** The `#` levels MinerU derived from `text_level` are read as a tree, not as decoration, so a paragraph deep in section 4.2.1 knows every heading above it.
2. **Tables stay whole.** Whether MinerU emitted a Markdown table or an HTML `<table>`, the table is treated as one unit and never cut mid-row. An embedding never sees half a table.
3. **Sentence-level chunks with lineage.** Every chunk carries `depth`, `chunk_index` and an embedding-ready `to_embed` form.
4. **Chunksets.** Chunks are grouped into [chunksets](/learn/chunking/chunksets): root-to-leaf paths through the hierarchy, so each retrieved passage explains itself. Because context comes from the path rather than from neighbouring text, no overlap is needed and the collection stays smaller.

Two honest limits. **Images are not fetched.** The `.md` references files under `images/`, and ingesting the Markdown alone does not follow those references, so figure content does not become searchable text. Caption them yourself first, or ingest the original PDF if figures carry answers. **Page numbers are not in the Markdown.** If you need per-chunk pages, join them from `content_list.json` via `page_idx`, or ingest the PDF.

On our reference legal document, chunksets answered the same query from **337 tokens** of retrieved context versus **1,542** for a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the [ingestion guide](/document-ingestion-chunking-rag) has the broader numbers.

## MinerU vs Docling and Marker

Pick on operational fit, not on chunking. All three are strong open-source parsers under active development, and all three produce structure a good chunker can use. MinerU stands out for formula and table handling and for offering pipeline, VLM and hybrid backends behind one CLI. Docling and Marker emit structured JSON with per-page blocks, which PrimeCut auto-detects — so with those two, page numbers survive into payloads without a manual join, whereas the MinerU route goes through Markdown. That is a difference in integration surface, not a verdict on parsing quality.

Decide on licence, hardware, language coverage, throughput and how the output looks on *your* documents. Then treat chunking as the separate problem it is. The [Docling](/optimal-chunker-docling) and [Marker](/optimal-chunker-marker) pages cover the equivalent ground.

## The MinerU RAG pipeline, end to end: MinerU to Qdrant {#end-to-end-mineru-to-qdrant}

Parse locally, chunk, store, retrieve.

```bash
pip install 'poma[qdrant]'
mineru -p contract.pdf -o out/ -b pipeline
```

```python
import glob
import os
from poma import PrimeCut
from poma.integrations.qdrant import PomaQdrant

# 1. Chunk the Markdown MinerU wrote.
md_path = glob.glob("out/contract/*/contract.md")[0]
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(md_path)

# 2. Qdrant — hybrid points with hierarchy payloads, one call.
qdrant = PomaQdrant(
    url=os.environ["QDRANT_URL"],
    api_key=os.environ["QDRANT_API_KEY"],
    cloud_inference=False,  # False embeds locally via fastembed, so OSS/local Qdrant works too
    collection_name="contracts",
    dense_model="sentence-transformers/all-MiniLM-L6-v2",
    sparse_model="Qdrant/bm25",
    dense_size=384,
    auto_create_collection=True,
)
qdrant.upsert_poma_points(result)

# 3. Retrieve prompt-ready context: hits merged along their shared heading path.
cheatsheets = qdrant.get_cheatsheets(
    query="What are the early termination conditions?",
    limit=3,
)
print(cheatsheets[0]["content"])
```

Nothing about your MinerU deployment changes: the parse runs on your hardware with your chosen backend, and the pipeline stays local except the chunking call. Any other vector store works the same way — embed `chunkset.to_embed` with your model and upsert it with `file_id` and `chunkset_index` in the payload.

## Chunking options for MinerU output, compared

| Approach | Structure used | Full hierarchy | Tables | Figures | Page numbers |
| --- | --- | --- | --- | --- | --- |
| Recursive character splitter | None | ✗ | Cut mid-row | Refs as noise | ✗ |
| Markdown-header splitter | Heading levels | ✗ | Usually survive | Refs as noise | ✗ |
| Custom `content_list.json` walker | Typed blocks | Manual | Manual | Manual | ✓ via `page_idx` |
| RAGFlow / RAG-Anything defaults | Framework-specific | ✗ | Varies | Varies | Varies |
| **PrimeCut on the Markdown** | **Full heading tree** | **✓ chunksets** | **Whole, never cut** | **Not fetched** | **Join from `page_idx`** |
| **PrimeCut on the original PDF** | **Full heading tree** | **✓ chunksets** | **Whole, never cut** | **Described** | **✓ per chunk** |

## Frequently asked questions

### How does MinerU chunking work for RAG?

Run MinerU as you do today, then chunk the Markdown file it writes. PrimeCut ingests that .md directly: it rebuilds the heading hierarchy from the # levels MinerU recovered, keeps tables whole, and returns chunks plus chunksets that are ready to embed. Avoid running a fixed-size character splitter over the Markdown, because that cuts through the headings and tables MinerU worked to reconstruct.

### Does MinerU chunk documents itself?

No. MinerU is a parser. It converts PDFs, images and Office files into Markdown plus JSON sidecars in reading order. It does not decide what one embedding vector should represent, and it produces no retrieval units. Chunking is left entirely to whatever you put downstream, which in practice is usually a generic text splitter.

### Should I chunk MinerU Markdown or content_list.json?

Chunk the Markdown, and keep the content list on disk. PrimeCut accepts the .md as a normal Markdown upload and rebuilds the hierarchy from it. The content_list.json is not one of the parser result shapes PrimeCut auto-detects, so it is not an ingest input, but it stays valuable as your own source of page_idx values, bounding boxes and block types for auditing, filtering and page-level joins.

### MinerU RAGFlow and RAG-Anything: how do they chunk its output?

Both use MinerU as the parser and then apply their own generic chunking. RAGFlow calls a MinerU API service as a remote client and offers MinerU as one PDF parser choice per dataset, after which its own chunking method splits the result. RAG-Anything selects MinerU with parser=mineru and then feeds the parsed content into its LightRAG-based pipeline. The parsing is layout-aware in both; the chunking that follows is not.

### What happens to MinerU images when I chunk the Markdown?

MinerU writes figures into an images directory and leaves relative references in the Markdown. If you ingest only the .md, those references are not followed and the image bytes are not fetched, so figures do not become searchable text. If figure content matters for retrieval, either caption the images yourself before chunking, or ingest the original PDF so the parsing and the figure handling happen in one pass.

### MinerU vs Docling vs Marker: which parser should I pick?

All three are strong open-source parsers and the honest answer is that the choice rarely decides retrieval quality. MinerU is notable for formula and table handling and for its pipeline, vlm and hybrid backends. Docling and Marker are also well-established and both produce structured JSON that PrimeCut auto-detects, while MinerU integration goes through its Markdown. Pick on licence, hardware, language coverage and speed, then spend the remaining effort on chunking.

### Do I have to re-parse the PDF to use PrimeCut with MinerU?

No. Point PrimeCut at the Markdown file MinerU already produced and only the chunking stages run. You keep your MinerU deployment, your backend choice and your GPU budget exactly as they are. Re-ingesting the original PDF is an option, not a requirement, and is mainly worth it when you need figures described.

### Which MinerU backend should I use for a RAG pipeline?

Use whichever backend already gives you the cleanest Markdown for your documents. The CLI exposes -b with pipeline, vlm-engine, hybrid-engine, vlm-http-client and hybrid-http-client, and -m with auto, txt and ocr for the pipeline path. Backend choice changes parsing accuracy and cost, not chunk quality; the same downstream chunking applies to every backend output.

## The rest of the series

The optimal chunker, for every OCR and parsing tool you already run:

- [AWS Textract](/optimal-chunker-textract)
- [Azure Document Intelligence](/optimal-chunker-azure-document-intelligence)
- [DeepSeek-OCR](/optimal-chunker-deepseek-ocr)
- [Docling](/optimal-chunker-docling)
- [LlamaParse](/optimal-chunker-llamaparse)
- [Marker](/optimal-chunker-marker)
- [Mistral OCR](/optimal-chunker-mistral-ocr)
- [PaddleOCR-VL](/optimal-chunker-paddleocr-vl)
- [Unstructured.io](/optimal-chunker-unstructured)

## Continue reading

- [RAG chunking strategies and text splitters](/rag-chunking-strategies-text-splitters) — the full taxonomy, and where each strategy breaks
- [POMA chunksets](/learn/chunking/chunksets) — what a root-to-leaf retrieval unit is and why it needs no overlap
- [The optimal chunker for Docling](/optimal-chunker-docling) — the same problem, with an auto-detected JSON result
- [The optimal chunker for Marker](/optimal-chunker-marker) — local parsing, block trees, and figure bytes
- [All pipeline recipes](/pipelines/) — parser to vector store, end to end