The Optimal Chunker for PaddleOCR-VL
The short answer: the optimal chunker for PaddleOCR-VL is one that reads its layout-parsing result directly — the markdown object when present, or the raw parsing_res_list block array otherwise — and turns PP-DocLayoutV3's own layout labels (doc_title, paragraph_title, table, running furniture) into real document hierarchy instead of flattening them into one text blob. POMA's bring-your-own-OCR connector auto-detects both shapes PaddleOCR-VL emits and returns hierarchy-preserving chunks and chunksets — no re-running OCR, no hand-rolled block-to-markdown converter.
This page covers what PaddleOCR-VL gives you as a self-hosted stack, the two shapes it can emit, and how to go from a layout-parsing result to embedded chunksets.
A self-hosted layout-parsing pipeline, not a cloud API
PaddleOCR-VL pairs a layout-detection model (PP-DocLayoutV3) with a vision-language OCR model, served behind an HTTP API you run yourself — typically via vLLM, with the layout server exposing the same /layout-parsing endpoint PaddleX's own serving stack uses. That matters for who reaches for it: teams that can't or won't send documents to a cloud OCR provider — data locality, air-gapped deployments, per-page cost at high volume — get real layout understanding without leaving their own infrastructure.
The pipeline's output reflects that layout-first design. Rather than returning a flat markdown string, PaddleOCR-VL labels every detected region:
{
"parsing_res_list": [
{"block_label": "doc_title", "block_content": "Annual Report", "block_id": 0, "block_order": 0},
{"block_label": "header", "block_content": "Confidential — Internal Use", "block_id": 1, "block_order": 1},
{"block_label": "paragraph_title", "block_content": "Financial highlights", "block_id": 2, "block_order": 2},
{"block_label": "table", "block_content": "<table>…</table>", "block_id": 3, "block_order": 3}
]
}Two shapes reach a bring-your-own-OCR connector, both are the tool's own output formats:
- The PaddleX serving envelope —
{"result": {"layoutParsingResults": [...]}}— where each page carries amarkdownobject (text+ a base64imagesside-dict) and aprunedResultholding the raw block list above. - A raw
save_to_json()page dict (or a list of them, one per PDF page) — the block list directly, with no markdown pre-assembly.
What PP-DocLayoutV3 and PaddleOCR-VL deliberately don't do: decide what a retrieval unit should be, or link a paragraph_title on page 12 back to the doc_title on page 1. That's the chunker's job — and it's the part a naive pipeline throws away.
Why flattening PaddleOCR-VL's labels wastes the layout model
The layout-detection step already did the hard classification work: it told you which region is a title, which is running furniture, which is a table. A pipeline that ignores this and just concatenates every block_content string — or worse, re-OCRs the flattened markdown with a generic splitter — discards exactly the structure you ran a layout model to get:
| What PP-DocLayoutV3 labeled | What naive flattening does |
|---|---|
doc_title / paragraph_title (heading levels) | Collapsed to plain text, indistinguishable from body |
header / footer / page_number (furniture) | Repeated on every page, polluting the index |
table (already HTML) | Sometimes re-parsed or split mid-row by downstream tooling |
block_order (true reading order) | Ignored if the caller sorts by anything else |
image / figure / chart with no inline bytes | Silently dropped — no signal that content is missing |
The result: a RAG pipeline that paid for layout detection and then threw the labels away before the LLM ever saw them.
The optimal chunker: read the labels, rebuild the hierarchy
POMA's bring-your-own-OCR connector handles both PaddleOCR-VL shapes:
- Prefers
markdown.textwhen present — already reading-ordered, tables already inline as HTML — and splicesmarkdown.images(a{filename: base64}side-dict) into![]()refs as data URIs so figures become describable. - Falls back to
parsing_res_listblock assembly when no markdown is present: blocks are sorted byblock_order(the model's true reading order, not array position);doc_titlebecomes#,paragraph_titlebecomes##;header,footer,page_number, andaside_text_numberlabels are dropped as running furniture;tableblocks pass through as inline HTML. - Handles all three image states: inline bytes/URLs become describable
![]()refs; an emptyimage/figure/chartblock becomes a visible[IMG-N]marker and is counted — never silently lost. - Maps
page_index(0-based,Noneonly for a bare single image) to 1-based page numbers consistently across both shapes. - Rebuilds cross-page hierarchy — a
paragraph_titleon page 12 is linked back to thedoc_titlefrom page 1 — and emits chunksets: root-to-leaf paths that keep every retrieved passage self-explanatory.
On one reference legal document, this hierarchy-preserving approach answered the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.
How to chunk a PaddleOCR-VL result with POMA
Run your self-hosted layout-parsing pipeline exactly as you do today, save the raw JSON response, and hand it to PrimeCut:
import json
import requests
from poma import PrimeCut
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. Chunk the raw result — POMA auto-detects the PaddleOCR-VL shape,
# serving envelope or bare save_to_json() page dicts alike.
client = PrimeCut() # reads POMA_API_KEY
result = client.ingest("contract.paddleocr-vl.json")
print(f"chunks: {len(result.chunks)}")
print(f"chunksets: {len(result.chunksets)}")
print(result.chunksets[0].to_embed)Detection fires on layoutParsingResults or parsing_res_list — both are PaddleOCR-VL's own key names, so a generic structured-JSON upload is never mistaken for one. To force or suppress it explicitly, set external_ocr_source to "paddleocr_vl" or "none".
Chunking options for PaddleOCR-VL output, compared
| Approach | Layout labels used | Cross-page hierarchy | Tables | Images | Page numbers kept |
|---|---|---|---|---|---|
| Concatenate blocks, character splitter | None | ✗ | Cut mid-row | Stripped or ignored | ✗ |
Concatenate markdown.text, header splitter | Partial (per-page) | ✗ | Usually survive | Usually stripped | Manual |
| POMA BYOCR (PrimeCut) | Full — titles, furniture, tables, order | ✓ chunksets | Inline HTML, never cut | Described or visibly counted | ✓ per chunk |
Frequently asked questions
How do I chunk PaddleOCR-VL output for RAG?
Save the raw layout-parsing result and feed it to a structure-aware chunker. POMA's BYOCR connector auto-detects PaddleOCR-VL's shape, assembles blocks in block_order, lifts title labels to real headings, and rebuilds hierarchy into chunks and chunksets.
Can I use a self-hosted PaddleOCR-VL result with POMA without sending documents to a cloud API?
Yes — the bring-your-own-OCR path doesn't care where OCR ran. Documents stay on your infrastructure through the OCR step; only the resulting JSON needs to reach POMA for chunking.
What does PaddleOCR-VL return, exactly?
Either the PaddleX serving envelope (result.layoutParsingResults[] with a markdown object and a prunedResult block list) or the raw save_to_json() output directly (parsing_res_list, an array of labeled blocks).
Does a recursive character splitter work on PaddleOCR-VL markdown?
It runs, but it discards the layout labels PP-DocLayoutV3 already assigned before splitting arbitrarily. Reading the block list directly keeps titles, furniture removal, and table integrity intact.
What happens to images in a PaddleOCR-VL result during chunking?
Inline image content (base64 or a URL) is described; empty image blocks become a visible [IMG-N] marker and are counted — never silently dropped.
Why self-host PaddleOCR-VL instead of using a cloud OCR API?
Data locality, air-gapped environments, and per-page cost at volume. It runs entirely on your infrastructure via vLLM behind an HTTP API; POMA's connector treats its output the same way it treats any cloud OCR result.
Ship the chunks to your vector database
- PaddleOCR-VL → Chroma
- PaddleOCR-VL → Elasticsearch
- PaddleOCR-VL → LanceDB
- PaddleOCR-VL → Milvus
- PaddleOCR-VL → MongoDB Atlas
- PaddleOCR-VL → OpenSearch
- PaddleOCR-VL → pgvector
- PaddleOCR-VL → Pinecone
- PaddleOCR-VL → Qdrant
- PaddleOCR-VL → Redis
- PaddleOCR-VL → Turbopuffer
- PaddleOCR-VL → Vespa
- PaddleOCR-VL → Weaviate
Or browse all pipeline recipes.
The rest of the series
- AWS Textract
- Azure Document Intelligence
- DeepSeek-OCR
- Docling
- LlamaParse
- Marker
- MinerU
- Mistral OCR
- Unstructured.io
Or start with the fundamentals: RAG chunking guide · Document ingestion guide · POMA chunksets.