PaddleOCR-VL to Chroma: Chunking for RAG
The short answer: PaddleOCR-VL gives you self-hosted, layout-aware OCR behind your own HTTP endpoint; Chroma gives you the fastest path from zero to working retrieval — an embedded, persistent vector store with metadata filtering and pluggable embedding functions. Wired together naively — flatten, split, embed — the prototype works just well enough to ship its worst decision everywhere, and inconsistent handling of PaddleOCR-VL's two accepted shapes across batch workers compounds it with a metadata schema that's incomplete for a silent subset of documents. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either shape, every time), emits hierarchy-preserving chunksets, and each lands in Chroma with file_id, page, depth, and chunkset_index metadata ready for where filters — with the .poma archive as a portable source of truth when the prototype graduates.
What each end of the pipeline actually provides
PaddleOCR-VL runs self-hosted — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a /layout-parsing HTTP API you operate. It returns either the PaddleX serving envelope (result.layoutParsingResults[], with a markdown object plus a prunedResult block list) or a raw save_to_json() page dict (parsing_res_list: flat blocks with block_label, block_content, block_id, block_order) — both legitimate, both liable to appear across a multi-worker batch pipeline. Details: The Optimal Chunker for PaddleOCR-VL.
Chroma provides a local-first store that runs embedded or client/server, collections with default or bring-your-own embedding functions, where filters over metadata, and where_document filters over content. What it doesn't provide: any opinion about what a document entry should contain, or metadata it wasn't given consistently. where={"file_id": ...} matches nothing for an entry that never received one. Details: The Optimal Chunks for the Best Retrieval in Chroma.
The naive wiring, and where it breaks
The common recipe — call /layout-parsing, hand-roll a block-to-text converter, split, collection.add(documents=windows) — breaks in a pair-specific way:
- Inconsistent shape handling ships an inconsistent schema. A self-hosted PaddleOCR-VL deployment commonly runs several batch workers; if one script only expects the serving envelope's
markdownobject and another only expects rawparsing_res_listdicts, the Chroma collection ends up with some entries carrying apageinteger and others carrying none —wherefilters then silently drop the incomplete subset with no error raised. - The prototype decision gets baked in. Chroma is where RAG pipelines are born; the chunking chosen in hour one is rarely revisited, so a hand-rolled flattener's gaps travel unnoticed into the production database it eventually ports to.
- Retrieved fragments arrive without lineage, so the LLM answers out of context — a failure that looks like "Chroma isn't good enough" when the actual defect is upstream, in how the two PaddleOCR-VL shapes were merged.
POMA's chunksets fix this at the source: one chunker, both shapes, the same hierarchy metadata every time. On a notoriously hard reference legal document, chunksets plus cheatsheet assembly delivered the answer in 337 tokens of retrieved context versus 1,542 for a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install requests poma chromadbimport json
import requests
import chromadb
from poma import PrimeCut
from poma.utils import unpack_poma_archive
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
# keep the .poma archive: page numbers live on its chunk records, not on the SDK objects
result = poma.ingest(
"contract.paddleocr-vl.json", # PaddleOCR-VL shape auto-detected
download_dir="store",
filename="contract.poma",
)
# Chunk lookups: depth from the SDK objects, page from the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
c["chunk_index"]: c.get("page")
for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}
# 3. Chroma — persistent local store; the collection's embedding
# function (default or your own) embeds the documents on add.
chroma = chromadb.PersistentClient(path="./chroma")
collection = chroma.get_or_create_collection(name="contracts")
# 4. Add chunksets — documents, stable IDs, where-filterable metadata.
leaves = [cs.chunks[-1] for cs in result.chunksets] # document order; last index is the leaf
collection.add(
ids=[f"{cs.file_id}:{cs.chunkset_index}" for cs in result.chunksets],
documents=[cs.to_embed for cs in result.chunksets],
metadatas=[
{
"file_id": cs.file_id,
"page": page_of[leaf],
"depth": depth_of[leaf],
"chunkset_index": cs.chunkset_index,
}
for cs, leaf in zip(result.chunksets, leaves)
],
)
# 5. Query with a metadata filter, then assemble the cheatsheet.
res = collection.query(
query_texts=["What are the early termination conditions?"],
n_results=5,
where={"file_id": "contract.paddleocr-vl.json"},
)
pairs = sorted(
zip(res["documents"][0], res["metadatas"][0]),
key=lambda p: p[1]["chunkset_index"],
)
cheatsheet = "\n\n".join(dict.fromkeys(doc for doc, _ in pairs))POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), prefers markdown.text when present and falls back to parsing_res_list sorted by block_order when it isn't — the same logic for every worker, every document — drops running furniture, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none".
Prototype today, port tomorrow
The .poma archive that PrimeCut produces is the portable source of truth for the document — chunks, chunksets, and hierarchy metadata, independent of any database and independent of which of PaddleOCR-VL's two shapes fed it. Loading it into Chroma is one loop; loading the same archive into Qdrant, Weaviate, or Pinecone is the same loop against a different client, because file_id/page/depth map onto every store's filtering primitive. The prototype and the production system disagree only about infrastructure, never about content — no re-running the self-hosted OCR call, no re-chunk, no drift between what the demo retrieved and what production retrieves.
Metadata mapping: POMA fields → Chroma primitives
| POMA chunk field | Chroma primitive | What it enables |
|---|---|---|
to_embed | documents entry, embedded by the collection's embedding function | semantic retrieval; where_document content filters |
file_id | metadatas key, where={"file_id": ...} | scope queries to one document |
page (on the archive's chunk records, from PaddleOCR-VL page_index, 1-based; take the chunkset's leaf chunk) | metadatas key, where with range operators | page-cited answers, page-range filters |
depth (of the chunkset's leaf chunk) | metadatas key | filter/re-rank by hierarchy level |
chunkset_index | metadatas key + part of the entry id | stable IDs, ordering, independent of block_id |
| chunkset lineage | .poma archive (portable source of truth) | identical reload into any other vector DB |
Frequently asked questions
How do I get self-hosted PaddleOCR-VL results into Chroma for RAG?
Save the raw /layout-parsing JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), then collection.add() with each chunkset's to_embed as the document and file_id/page/depth/chunkset_index metadata. page comes from the archive's chunk records (the chunkset's leaf chunk) — the SDK objects expose depth but not page. Query with where filters and assemble cheatsheets client-side.
What goes wrong when different workers process PaddleOCR-VL's two shapes inconsistently?
If one worker's script only handles the serving envelope and another only handles raw save_to_json dicts, the resulting collection has a schema gap — some entries lack page — and where filters silently exclude that subset without an error.
What metadata should Chroma store for PaddleOCR-VL chunks so where filters work?
file_id, page (from page_index, read off the archive's chunk records), depth, and chunkset_index. POMA writes all four consistently no matter which of the two accepted shapes produced a given chunk.
Will a Chroma prototype built on self-hosted PaddleOCR-VL chunks port to a production vector database?
The .poma archive reloads into Qdrant, Weaviate, or Pinecone without re-running OCR or re-chunking. A hand-rolled flattener's schema gaps also port — which is why routing both shapes through one chunker from day one matters.
Does staying self-hosted through Chroma matter for data locality?
Yes — PaddleOCR-VL and Chroma can both run entirely on infrastructure you control. Only the layout-parsing result JSON needs to reach POMA's chunking API.
Related recipes
Same parser, different store: PaddleOCR-VL → Qdrant · PaddleOCR-VL → Pinecone · PaddleOCR-VL → Weaviate
Same store, different parser: Mistral OCR → Chroma · LlamaParse → Chroma · Docling → Chroma
Foundations: The Optimal Chunker for PaddleOCR-VL · The Optimal Chunks for Chroma · All pipeline recipes · RAG chunking guide