PaddleOCR-VL to Milvus: Chunking for RAG
The short answer: PaddleOCR-VL gives you self-hosted, layout-labeled document parsing; Milvus gives you filtered vector search at serious scale — partitions, scalar filter expressions, and your pick of HNSW, IVF, or DiskANN. Wired together naively — flatten the labeled blocks, split, embed, insert — the pipeline underuses both, because nothing in between rebuilds the document hierarchy or populates the scalar fields Milvus filters on. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either PaddleOCR-VL shape), emits hierarchy-preserving chunksets, and each chunkset lands in Milvus with file_id, page, depth, and chunkset_index as scalar fields ready for filter expressions and partition-key routing.
What each end of the pipeline actually provides
PaddleOCR-VL runs entirely on your own infrastructure — a layout-detection model (PP-DocLayoutV3) plus a vision-language OCR model served via vLLM behind a PaddleX-compatible /layout-parsing endpoint. It returns a serving envelope with a pre-assembled markdown object, or the raw save_to_json() block list (parsing_res_list) of {block_label, block_content, block_order} entries. What it doesn't provide: a link between a heading on one page and its ancestor on another, or any opinion about what a retrieval unit should be. Details: The Optimal Chunker for PaddleOCR-VL.
Milvus provides collections with typed schemas, partitions with automatic partition-key routing for multi-tenancy, scalar fields queried through filter expressions (file_id == "...", page <= 10), index types from in-memory HNSW through IVF to disk-resident DiskANN, and — in recent versions — sparse vectors with built-in BM25 for hybrid search. What it doesn't provide: any opinion about what a row should contain. Empty scalar fields filter nothing; fragment vectors retrieve fragments. Details: The Optimal Chunks for the Best Retrieval in Milvus.
The naive wiring, and where it breaks
The common recipe — concatenate parsing_res_list block content (or markdown.text pages) → RecursiveCharacterTextSplitter → embed → client.insert(...) — breaks in a pair-specific way:
page_indexisNonefor a bare single image, not a PDF page. A naive pipeline that inserts it directly into Milvus's INT64pagefield either fails the insert on a non-nullable column, or the client coercesNoneto0— which then collides with legitimate index-0 content and quietly corrupts everypage <= Nfilter expression built for page-cited answers.- The scalar fields end up empty otherwise. A flat splitter emits anonymous text windows: no page (the join across pages discarded
page_index), no depth (PP-DocLayoutV3'sdoc_title/paragraph_titlelabels were collapsed to plain text), often no reliable document ID. Milvus's headline feature, filtered search, has nothing to filter on. - Overlap bloats the index. Splitter overlap embeds every boundary span twice, inflating vector count across the board — and it hits DiskANN hardest, where a disk-resident index pays for near-duplicate vectors in build time and disk footprint.
POMA's chunksets fix all three at the source: page_index is normalized to a 1-based page number (or left unset, never coerced to a colliding zero), hierarchy metadata comes from PP-DocLayoutV3's own labels, and there's no overlap. On a notoriously hard reference legal document, chunksets plus cheatsheet assembly delivered the answer in 337 tokens of retrieved context versus 1,542 for a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install poma requests pymilvus sentence-transformersimport json
import os
import requests
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from pymilvus import DataType, MilvusClient
from sentence_transformers import SentenceTransformer
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
# keep the .poma archive: page numbers live on its chunk records, not on the SDK objects
result = poma.ingest(
"contract.paddleocr-vl.json", # PaddleOCR-VL shape auto-detected
download_dir="store",
filename="contract.poma",
)
# Chunk lookups: depth from the SDK objects, page from the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
c["chunk_index"]: c.get("page")
for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}
# 3. Milvus — a schema whose scalar fields mirror the hierarchy metadata.
client = MilvusClient(uri=os.environ["MILVUS_URI"])
schema = MilvusClient.create_schema()
schema.add_field("pk", DataType.INT64, is_primary=True, auto_id=True)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=384)
schema.add_field("text", DataType.VARCHAR, max_length=65535)
schema.add_field("file_id", DataType.VARCHAR, max_length=256, is_partition_key=True)
schema.add_field("page", DataType.INT64)
schema.add_field("depth", DataType.INT64)
schema.add_field("chunkset_index", DataType.INT64)
index_params = client.prepare_index_params()
index_params.add_index(field_name="vector", index_type="HNSW", metric_type="COSINE")
client.create_collection("contracts", schema=schema, index_params=index_params)
# 4. Embed chunksets and insert — hierarchy metadata rides along as scalars.
# The archive's page is None for a bare single image; coerce explicitly
# rather than letting it collide with real page-0 content.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
leaves = [cs.chunks[-1] for cs in result.chunksets] # document order; last index is the leaf
client.insert(
collection_name="contracts",
data=[
{
"vector": model.encode(cs.to_embed).tolist(),
"text": cs.to_embed,
"file_id": cs.file_id,
"page": page_of[leaf] if page_of[leaf] is not None else -1,
"depth": depth_of[leaf],
"chunkset_index": cs.chunkset_index,
}
for cs, leaf in zip(result.chunksets, leaves)
],
)
# 5. Filtered search — scope to one document, then assemble the cheatsheet.
query = "What are the early termination conditions?"
hits = client.search(
collection_name="contracts",
data=[model.encode(query).tolist()],
limit=5,
filter='file_id == "contract.paddleocr-vl.json"',
output_fields=["text", "page", "depth", "chunkset_index"],
)
ordered = sorted(hits[0], key=lambda h: h["entity"]["chunkset_index"])
cheatsheet = "\n\n".join(dict.fromkeys(h["entity"]["text"] for h in ordered))POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles both PaddleOCR-VL shapes, assembles blocks by block_order rather than array position, drops running furniture (header/footer/page_number), and normalizes page_index to 1-based page numbers before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none". The final step is the cheatsheet: retrieved chunksets are deduplicated (shared ancestor headings appear once) and merged into a single prompt-ready block — Milvus returns the rows, your client assembles them.
Metadata mapping: POMA fields → Milvus primitives
| POMA chunk field | Milvus primitive | What it enables |
|---|---|---|
to_embed | FLOAT_VECTOR field (add a sparse/BM25 field for hybrid) | paraphrase + exact-term retrieval |
file_id | VARCHAR scalar field, partition key | per-document filters, automatic multi-tenant routing |
page (on the archive's chunk records — PaddleOCR-VL's page_index, 0-based, normalized to 1-based; None for a bare image; take the chunkset's leaf chunk) | INT64 scalar field | page-cited answers, page <= N filter expressions |
depth (of the chunkset's leaf chunk, from doc_title/paragraph_title label levels) | INT64 scalar field | filter/re-rank by hierarchy level |
chunkset_index (document order) | INT64 scalar field | stable, correctly-sequenced ordering at assembly |
| chunkset text | VARCHAR field in output_fields | assembly without a second store |
Frequently asked questions
How do I get self-hosted PaddleOCR-VL results into Milvus for RAG?
Save the raw /layout-parsing response, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), embed each chunkset's to_embed, and insert rows whose scalar fields carry file_id, page, depth, and chunkset_index. page comes from the archive's chunk records (the chunkset's leaf chunk) — the SDK objects expose depth but not page. Retrieve with a filtered search and assemble cheatsheets client-side.
Why does PaddleOCR-VL's page_index need special handling before it reaches Milvus?
page_index is None for a bare single image. Inserted directly, it either fails a non-nullable INT64 field or coerces to 0, colliding with real page-0 content in page <= N filters. POMA normalizes it to a 1-based number, or leaves it explicitly unset.
What scalar fields should a Milvus collection have for PaddleOCR-VL chunks?
file_id (VARCHAR, the partition-key candidate), page (INT64, normalized from page_index, read off the archive's chunk records), depth (INT64, from PP-DocLayoutV3's title labels), and chunkset_index (INT64, document order) alongside the vector and text.
Should I use a Milvus partition key for multi-tenant self-hosted PaddleOCR-VL content?
Yes — declare file_id as the partition key so filtered searches only touch matching partitions. POMA populates file_id on every chunkset from either PaddleOCR-VL output shape, so the key always has a value.
Does Milvus hybrid sparse plus BM25 search help with self-hosted PaddleOCR-VL documents?
Yes — self-hosted PaddleOCR-VL pipelines often handle documents dense with exact tokens (part codes, clause numbers) that dense embeddings blur. Recent Milvus versions store sparse BM25 vectors next to dense ones and fuse both at query time.
Related recipes
Same parser, different store: PaddleOCR-VL → pgvector · PaddleOCR-VL → Turbopuffer · PaddleOCR-VL → Vespa
Same store, different parser: Mistral OCR → Milvus · Unstructured.io → Milvus · Azure Document Intelligence → Milvus
Foundations: The Optimal Chunker for PaddleOCR-VL · The Optimal Chunks for Milvus · All pipeline recipes · RAG chunking guide