Source: http://www.poma-ai.com/docs/optimal-chunker-marker

# The Optimal Chunker for Marker

<ByAuthor />

**The short answer:** the optimal chunker for Marker is one that consumes Marker's raw output *as-is* — ideally the JSON renderer's Document block tree — preserves the per-page structure and full HTML tables Marker recovered, splices the side-channel image bytes back in, rebuilds the heading hierarchy across pages, and emits retrieval units that carry their own context. POMA's bring-your-own-OCR connector does exactly this: upload the saved Marker result, and PrimeCut returns hierarchy-preserving **chunks** and **chunksets** — no re-parsing, no markdown flattening, no hand-rolled splitter.

This page explains what Marker gives you, where it stops, why generic text splitters squander it, and how to go from a `marker_single` run to embedded chunksets in a few lines.

## What Marker gives you — and where it stops

Marker (by Datalab / VikParuchuri) has earned its place as the fast open-source PDF→markdown favourite for local pipelines: it runs on your own hardware, keeps documents on your machine, and produces clean markdown from layouts that break naive extractors. Depending on how you invoke it, the output takes one of a few shapes:

- **MarkdownOutput** — the default: `{markdown, images, metadata}`, one markdown string for the document plus a side-channel `images` dict of `{name: base64}` and run metadata.
- **The `{output_format, …}` envelope** — the shape you get from Marker's serving layer, carrying the rendered output alongside its declared format.
- **The deprecated `{output, format}` envelope** — older Marker versions; still seen in saved results.
- **The JSON renderer's Document block tree** (`--output_format json`) — a root block with `block_type: "Document"` whose children are Pages; every block embeds its content as HTML, and tables arrive as full `<table>` elements with row and column spans intact.

Within a page, structure is recovered: headings, lists, and tables. That is real, hard-won structure — and it's exactly the part most pipelines then throw away.

What Marker deliberately does **not** do:

- **Cross-page hierarchy.** A `## Termination clauses` heading on page 41 has no machine-readable link to the `# Master Services Agreement` chapter that opened on page 3. The markdown output doesn't even keep page boundaries.
- **Chunking.** Marker's output is display markdown, not retrieval units. Deciding what an embedding vector should represent is left entirely to you.
- **Image semantics.** Extracted figures live in the side-channel `images` dict; the text carries only `![](name)` references. Something downstream has to reunite bytes with references and make figures searchable — or the figures are simply gone from your index.

Marker's job ends at "here is markdown." Display markdown and retrieval units are different artifacts, and the gap between them is where local RAG pipelines quietly fail.

## Why generic text splitters waste Marker output

The default move — take `rendered.markdown`, run `RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)` — destroys most of what Marker just computed on your GPU:

| What Marker recovered | What a character splitter does with it |
| --- | --- |
| Heading levels (`#`, `##`, `###`) | Ignored — cuts fall wherever the character count lands |
| HTML tables with row/colspans | Sliced mid-row; the span structure is meaningless in a fragment |
| Page blocks (JSON renderer) | Never consulted — most pipelines don't even request the JSON output |
| Side-channel `images` dict | Left behind entirely; `![](name)` refs point at nothing |
| Reading order Marker reconstructed | Preserved only by luck of the cut positions |

Markdown-header splitters (splitting on `##` boundaries) are a step up, but they still produce **isolated fragments**: the chunk under `### Early termination` no longer knows it lives inside `## Termination clauses` inside `# Master Services Agreement`. At retrieval time the LLM reads an orphaned paragraph and answers out of context. That failure mode — not parsing accuracy — is where most Marker-based RAG pipelines lose their quality.

For the full taxonomy of these failure modes, see the [RAG chunking strategies guide](/rag-chunking-strategies-text-splitters).

## The optimal chunker: hierarchy-aware chunking on the raw result

POMA's **bring-your-own-OCR** (BYOCR) connector was built for teams who already run their own parsing step — which describes almost every Marker user, since running locally is the point. The connector consumes the *saved* Marker result and runs only POMA's downstream stages on it:

1. **Shape validation, up front.** The JSON renderer's Document block tree is the strict auto-route fingerprint: a root `block_type: "Document"` with Page children is routed through the BYOCR seam automatically. The bare markdown shapes (`{markdown, images, metadata}`, `{output_format, …}`, the deprecated `{output, format}`) are too generic to fingerprint safely, so they are accepted only on the connector endpoint or with an explicit `external_ocr_source: "marker"` declaration. A corrupted or mislabeled upload fails immediately with a clear 422 — never a silently degraded index.
2. **Block tree → per-page markdown.** For JSON output, POMA walks the Pages and converts each block's embedded HTML to markdown per page — so page boundaries survive even though Marker's markdown output alone would have lost them.
3. **Image splicing.** The side-channel `images` dict (`{name: base64}`) is spliced into the matching `![](name)` references as data URIs, and POMA's image-description pass turns each figure into searchable text — figures get described like inline images instead of lost. Any reference without bytes is neutralized and **counted in `content_metadata`**: loss is surfaced, never silent.
4. **Table preservation.** Marker's full `<table>` elements — row and column spans included — are carried into the chunk layer as HTML, so embeddings never see half a table.
5. **Noise removal.** Running headers and footers that repeat on every page are stripped — the same pass POMA's native pipeline applies.
6. **Heading enrichment.** Where Marker's output under-marks headings (design-heavy or scanned sources), POMA's heading-injection pass restores them before chunking — the same treatment every natively ingested document gets.
7. **Hierarchy → chunks + chunksets.** The cross-page heading tree is rebuilt, and every sentence is emitted as a chunk with its `depth`, `page`, and a `to_embed` normalization — grouped into [chunksets](/learn/chunking/chunksets): root-to-leaf paths that keep each retrieved passage self-explanatory.

Because you already ran the parsing on your own hardware, **POMA charges only for the downstream structure and chunking value** — the OCR front-end is skipped entirely.

The result, measured on our reference legal-document benchmark: the same query answered with **337 tokens** of retrieved context instead of **1,542** for a recursive character splitter, with zero information loss. Methodology: [Document Ingestion & Chunking for RAG](/document-ingestion-chunking-rag).

## How to chunk Marker output with POMA

Run Marker exactly as you do today — the only recommendation is the JSON renderer, so the block tree survives — and hand the saved result to PrimeCut:

```bash
# 1. Your existing Marker run — unchanged. The JSON renderer keeps the
#    Document block tree that POMA auto-detects.
marker_single contract.pdf --output_format json --output_dir out/
```

```python
from poma import PrimeCut

# 2. Chunk the raw result — the Document block tree auto-routes.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("out/contract/contract.json")

print(f"chunks: {len(result.chunks)}")
print(f"chunksets: {len(result.chunksets)}")
print(result.chunksets[0].to_embed)  # embedding-ready, hierarchy included
```

The same works from Marker's Python converter API — render with the JSON renderer, save the result, ingest the file.

If what you have is one of the **bare markdown shapes** — `{markdown, images, metadata}` from a default run, or the deprecated `{output, format}` envelope — declare the source explicitly, because those shapes are not auto-routed:

```python
result = client.ingest("contract.marker.json", external_ocr_source="marker")
```

Two things worth knowing:

- **Strict auto-detection, by design.** Only the JSON Document block tree fingerprints as Marker. A two-or-three-key markdown wrapper could be anything, and misrouting someone's hand-rolled JSON would be worse than asking for a declaration — so bare markdown shapes require the explicit `external_ocr_source: "marker"` parameter or the connector endpoint. To opt a look-alike JSON *out* of detection entirely, set `external_ocr_source: "none"`.
- **Dedicated endpoint.** For explicit routing, the API exposes a `chunk_external_ocr_result` endpoint that accepts `external_ocr_source: "marker"`. See the [API reference](https://api.poma-ai.com/v3/docs).

From `result.chunks` / `result.chunksets` onward, everything is your stack: your vector store, your embedding model, your retrieval strategy.

## Chunking options for Marker output, compared

| Approach | Structure used | Cross-page hierarchy | Tables | Images | Page numbers kept |
| --- | --- | --- | --- | --- | --- |
| Recursive character splitter | None | ✗ | Cut mid-row, spans destroyed | Dangling `![](name)` refs | ✗ |
| Markdown-header splitter | Flat heading marks | ✗ | Usually survive | Refs stripped, bytes left behind | ✗ |
| Semantic chunker | Embedding similarity | ✗ | Fragile | Stripped | ✗ |
| **POMA BYOCR (PrimeCut)** | **Full block tree + heading tree, rebuilt across pages** | **✓ chunksets** | **Kept as HTML, spans intact, never cut** | **Spliced from the images dict and described** | **✓ per chunk (JSON renderer)** |

## Frequently asked questions

### How do I chunk Marker output for RAG?

Keep the structure Marker recovered instead of flattening it. Run Marker with the JSON renderer (`--output_format json`) so the Document block tree survives, then feed the saved result to a structure-aware chunker. POMA's BYOCR connector accepts the raw Marker JSON, converts each page's block HTML to markdown, rebuilds the heading hierarchy across pages, and emits chunks plus chunksets ready for embedding.

### Can I chunk Marker results with POMA without re-parsing the PDF?

Yes — that's the point of the bring-your-own-OCR path. POMA skips its OCR front-end and runs only the downstream structure, chunking, and retrieval-preparation stages on your result. The Document block tree is auto-detected; bare markdown shapes need `external_ocr_source: "marker"` or the connector endpoint.

### Which Marker output format is best for RAG chunking — markdown or JSON?

JSON. The default markdown output is one flat display string; the JSON renderer preserves per-page blocks, explicit block types, and full HTML tables with row and column spans — exactly the structure a hierarchy-preserving chunker needs. Markdown output still works, but declare `external_ocr_source` explicitly.

### What happens to images in Marker output during chunking?

Marker parks figure bytes in a side-channel `images` dict and leaves `![](name)` refs in the text. POMA splices the bytes back in as data URIs and describes each figure, so images become searchable text. Refs without bytes are neutralized and counted in `content_metadata` — loss is always visible.

### Does POMA auto-detect Marker output?

The JSON Document block tree auto-routes. The bare markdown shapes (`{markdown, images, metadata}`, deprecated `{output, format}`) are deliberately not fingerprinted — a small markdown wrapper is too generic — so those require the explicit declaration or the `chunk_external_ocr_result` endpoint.

### Do Marker's tables survive chunking with POMA?

Yes. Marker's JSON output carries tables as full HTML `<table>` elements with row and column spans; POMA keeps them as HTML through the chunk layer, so an embedding never sees half a table. Character splitters routinely cut them mid-row.

## The rest of the series

The optimal chunker, for every OCR and parsing tool you already run:

- [Mistral OCR](/optimal-chunker-mistral-ocr)
- [LlamaParse](/optimal-chunker-llamaparse)
- [Azure Document Intelligence](/optimal-chunker-azure-document-intelligence)
- [Unstructured.io](/optimal-chunker-unstructured)
- [Docling](/optimal-chunker-docling)
- [AWS Textract](/optimal-chunker-textract)

Wire Marker straight into your vector database:

- [Marker → Qdrant](/pipelines/marker-to-qdrant)
- [Marker → Pinecone](/pipelines/marker-to-pinecone)
- [Marker → Weaviate](/pipelines/marker-to-weaviate)
- [Marker → pgvector](/pipelines/marker-to-pgvector)
- [Marker → Milvus](/pipelines/marker-to-milvus)
- [Marker → Chroma](/pipelines/marker-to-chroma)

Or start with the fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [Document ingestion guide](/guides/document-ingestion/) · [POMA chunksets](/learn/chunking/chunksets).