PaddleOCR-VL to Qdrant: Chunking for RAG
The short answer: PaddleOCR-VL gives you self-hosted, layout-aware OCR — PP-DocLayoutV3 labels every region, PaddleOCR-VL reads it, all behind your own HTTP endpoint. Qdrant gives you excellent hybrid dense+sparse vector search. Wired together naively — flatten, split, embed — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy or respects PaddleOCR-VL's own reading-order field. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either accepted shape), emits hierarchy-preserving chunksets, and PomaQdrant.upsert_poma_points() writes them as hybrid dense+sparse points that carry the chunkset's lineage in the payload.
What each end of the pipeline actually provides
PaddleOCR-VL runs as a self-hosted pipeline — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a /layout-parsing HTTP API you operate yourself. It returns either the PaddleX serving envelope (result.layoutParsingResults[], each with a markdown object and a prunedResult block list) or a raw save_to_json() page dict with parsing_res_list: flat blocks carrying block_label, block_content, block_id, and block_order. What it doesn't provide: an opinion on what a retrieval unit should be, or a link between a paragraph_title on page 12 and the doc_title on page 1. Details: The Optimal Chunker for PaddleOCR-VL.
Qdrant provides named dense + sparse vectors on one point, payload indexes for filtered ANN, and HNSW with quantization options — open source or Qdrant Cloud. What it doesn't provide: any opinion about what a point should contain. It retrieves nearest neighbors of whatever you embedded, including scrambled or context-free fragments, if that's what you gave it. Details: The Optimal Chunks for the Best Retrieval in Qdrant.
The naive wiring, and where it breaks
The common recipe — call /layout-parsing, concatenate block_content (sorted by block_id, the field that happens to come first in the JSON) or join markdown.text across pages, run a RecursiveCharacterTextSplitter, embed, client.upsert(...) — breaks in a pair-specific way:
- Sorting by
block_idinstead ofblock_orderscrambles reading order.block_idis assignment order, not reading order; on a multi-column page the two diverge. A pipeline that sorts the wrong field embeds a logically reordered document, and Qdrant's BM25 sparse vectors then represent clause sequences that never appeared in that order in the source. - PaddleOCR-VL's
page_indexvanishes at the join, so no Qdrant payload can cite a page, and per-document filters are all that's left. - Overlap inflates the collection. Splitter overlap embeds every boundary span twice, so top-k results arrive as near-duplicates of each other, crowding out the passage that actually answers the question — regardless of how clean the underlying OCR was.
Chunksets fix this at the unit level: block_order-respecting assembly, no overlap, page and hierarchy metadata kept on the chunk records in the .poma archive. On a reference legal document, this approach answered the same query with 337 tokens of retrieved context versus 1,542 for a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install requests 'poma[qdrant]'import json
import os
import requests
from poma import PrimeCut
from poma.integrations.qdrant import PomaQdrant
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest("contract.paddleocr-vl.json") # PaddleOCR-VL shape auto-detected
# 3. Qdrant — hybrid points with hierarchy payloads, one call.
qdrant = PomaQdrant(
url=os.environ["QDRANT_URL"],
api_key=os.environ["QDRANT_API_KEY"],
cloud_inference=False, # True needs Qdrant Cloud's inference service; False embeds locally via fastembed and works on OSS/local Qdrant too
collection_name="contracts",
dense_model="sentence-transformers/all-MiniLM-L6-v2",
sparse_model="Qdrant/bm25",
dense_size=384,
auto_create_collection=True,
)
qdrant.upsert_poma_points(result)
# 4. Retrieve prompt-ready context.
cheatsheets = qdrant.get_cheatsheets(
query="What are the early termination conditions?",
limit=3,
)
print(cheatsheets[0]["content"])POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), prefers markdown.text when present and falls back to parsing_res_list sorted by block_order when it isn't, drops running furniture (header/footer/page_number/aside_text_number), and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none".
Metadata mapping: POMA fields → Qdrant primitives
One point per chunkset. upsert_poma_points writes exactly this payload: {"chunkset_index", "chunks", "file_id", "text"} plus "chunk_details" (the chunkset's chunk records — chunk_index, content, depth, file_id, code) while store_chunk_details stays at its default True.
| POMA field | Qdrant primitive | What it enables |
|---|---|---|
to_embed | dense vector + BM25 sparse vector (named vectors, one point) | paraphrase + exact-term hybrid retrieval |
text (a copy of to_embed) | payload field | read back what was embedded |
file_id | payload field (no index is created for you — add one with create_payload_index) | scope queries to one document |
chunkset_index | payload field | identifies the point: the id is a UUIDv5 of file_id + chunkset_index |
chunks (chunk indices) | payload field | joins the point back to its chunk records, independent of block_id |
chunk lineage (chunk_index, content, depth, code) | payload (chunk_details) | cheatsheet assembly without a second store |
page (from PaddleOCR-VL page_index, 1-based) | not written by PomaQdrant — the SDK's PomaChunk drops it; it survives on the archive's chunk records | page-cited answers, page-range filters — see below |
get_cheatsheets reads only that payload back: it groups each hit's chunkset_index + chunks by file_id and assembles the text from the hit's chunk_details (or from a chunk_data you pass instead).
Page numbers are in the .poma archive, so keep it (poma.ingest(..., download_dir="store", filename="contract.poma")) and attach them to the same points:
from poma.utils import unpack_poma_archive
from poma.integrations.qdrant.qdrant_poma_utils import chunk_uuid_string
page_of = {c["chunk_index"]: c.get("page") for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]}
for cs in result.chunksets:
leaf = cs.chunks[-1] # chunk indices are in document order; the last one is the leaf
qdrant.set_payload(collection_name="contracts", payload={"page": page_of[leaf]}, points=[chunk_uuid_string(cs.file_id, cs.chunkset_index)])Frequently asked questions
How do I get self-hosted PaddleOCR-VL results into Qdrant for RAG?
Save the raw /layout-parsing JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), then PomaQdrant.upsert_poma_points(result) — hybrid points whose payload carries file_id, chunkset_index, the chunkset's chunk indices, the embedded text, and chunk_details (chunk_index, content, depth) for assembly. Retrieve with get_cheatsheets(query=...).
Why does block_order matter more than block_id when feeding Qdrant's hybrid search?
block_id is detection order, not reading order; block_order is. Sorting by the wrong field scrambles multi-column text before it's embedded, so Qdrant's BM25 sparse vectors represent clause sequences that never occurred in that order in the source.
What Qdrant payload fields should PaddleOCR-VL chunks carry?
PomaQdrant writes file_id, chunkset_index, chunks, text and chunk_details (the chunk records with chunk_index, content and depth) by default, regardless of which accepted PaddleOCR-VL shape produced them. It does not write page, and it creates no payload indexes: add a payload index on file_id yourself, and if you need page-cited answers or page-range filters, read page off the archive's chunk records with unpack_poma_archive and attach it with client.set_payload on the same point ids.
Can I keep documents on my own infrastructure through both OCR and retrieval?
Yes — PaddleOCR-VL and Qdrant can both run self-hosted. Only the layout-parsing result JSON needs to reach POMA's API for chunking; the source document and the vector index never have to leave your infrastructure.
Do images in the PaddleOCR-VL result survive the trip to Qdrant?
Yes, when present: inline base64 or referenced bytes are described and embedded as searchable text. Empty image blocks become a visible [IMG-N] marker and are counted — never silently dropped.
Related recipes
Same parser, different store: PaddleOCR-VL → Pinecone · PaddleOCR-VL → Weaviate · PaddleOCR-VL → Chroma
Same store, different parser: Mistral OCR → Qdrant · LlamaParse → Qdrant · Docling → Qdrant
Foundations: The Optimal Chunker for PaddleOCR-VL · The Optimal Chunks for Qdrant · All pipeline recipes · RAG architecture guide