PaddleOCR-VL to Pinecone: Chunking for RAG
The short answer: PaddleOCR-VL gives you self-hosted, layout-aware OCR you run behind your own HTTP endpoint; Pinecone gives you excellent serverless filtered ANN with namespaces. Wired together naively — flatten, split, embed, stuff everything into metadata for self-description — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy, and Pinecone's ~40 KB metadata cap punishes the workaround. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either accepted shape), emits hierarchy-preserving chunksets, and you upsert their embeddings with compact {file_id, page, depth} metadata — full text stays in your doc store or the .poma archive.
What each end of the pipeline actually provides
PaddleOCR-VL runs self-hosted — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a /layout-parsing HTTP API you operate. It returns either the PaddleX serving envelope (result.layoutParsingResults[], with a markdown object plus a prunedResult block list) or a raw save_to_json() page dict (parsing_res_list: flat blocks with block_label, block_content, block_id, block_order). What it doesn't provide: any retrieval unit, or a link between a heading on one page and its ancestor on another. Details: The Optimal Chunker for PaddleOCR-VL.
Pinecone provides serverless indexes, namespaces for hard tenant partitioning, metadata filtering ($eq, $in, $gte, …) under a ~40 KB per-vector metadata limit, and sparse-dense hybrid support. What it doesn't provide: any opinion about what a vector should represent, or a place for long text — metadata is a filter mechanism, not a document store. Details: The Optimal Chunks for the Best Retrieval in Pinecone.
The naive wiring, and where it breaks
The common recipe — call /layout-parsing, concatenate blocks or markdown.text across pages, split, embed, index.upsert(...) — breaks in a pair-specific way:
- The metadata cap collides with PaddleOCR-VL's raw shape. To make a Pinecone hit self-describing without a second store, teams stuff the
parsing_res_listblock array — or the assembled block text plus lineage — into metadata, and hit the ~40 KB cap on exactly the long, multi-column, block-dense pages PaddleOCR-VL's layout model was chosen to handle. page_indexvanishes at the join, so no metadata field can cite a page and$gte/$ltepage filters have nothing to act on.- Overlap inflates the namespace. Splitter overlap embeds every boundary span twice, so top-k results arrive as near-duplicates crowding out the passage that actually answers the question.
Chunksets fix this at the unit level: each is a self-explanatory root-to-leaf path, so the vector needs only compact filter metadata, not a smuggled block array. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install requests poma pinecone sentence-transformersimport json
import os
import requests
from pinecone import Pinecone, ServerlessSpec
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
# keep the .poma archive: page numbers live on its chunk records, not on the SDK objects
result = poma.ingest(
"contract.paddleocr-vl.json", # PaddleOCR-VL shape auto-detected
download_dir="store",
filename="contract.poma",
)
# Chunk lookups: depth from the SDK objects, page from the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
c["chunk_index"]: c.get("page")
for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}
# 3. Embed chunksets; upsert with COMPACT metadata (text lives outside Pinecone).
model = SentenceTransformer("all-MiniLM-L6-v2")
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
pc.create_index(
name="contracts", dimension=384, metric="cosine",
spec=ServerlessSpec(cloud="aws", region="us-east-1"),
)
index = pc.Index("contracts")
file_id = result.chunks[0].file_id
store = {} # your doc store — or resolve IDs from the .poma archive
vectors = []
for cs in result.chunksets:
leaf = cs.chunks[-1] # document order; the last chunk index is the leaf
vec_id = f"{file_id}:{cs.chunkset_index}"
store[vec_id] = cs.to_embed
vectors.append({
"id": vec_id,
"values": model.encode(cs.to_embed).tolist(),
"metadata": {"file_id": file_id, "page": page_of[leaf],
"depth": depth_of[leaf], "chunkset_index": cs.chunkset_index},
})
index.upsert(vectors=vectors, namespace="acme-corp")
# 4. Filtered query, then cheatsheet assembly from the doc store.
q = model.encode("What are the early termination conditions?").tolist()
res = index.query(
vector=q, top_k=3, namespace="acme-corp",
filter={"file_id": {"$eq": file_id}}, include_metadata=True,
)
cheatsheet = "\n\n".join(store[m["id"]] for m in res["matches"])POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), prefers markdown.text when present and falls back to parsing_res_list sorted by block_order when it isn't, drops running furniture, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none".
Metadata mapping: POMA fields → Pinecone primitives
| POMA chunk field | Pinecone primitive | What it enables |
|---|---|---|
to_embed | dense values (your embedding model) | paraphrase retrieval |
to_embed tokens | optional sparse values on the same record | exact-term hybrid retrieval |
file_id | metadata, $eq filter | scope queries to one document |
page (on the archive's chunk records, from PaddleOCR-VL page_index, 1-based; take the chunkset's leaf chunk) | metadata, $gte/$lte filters | page-cited answers, page-range scoping |
depth (of the chunkset's leaf chunk) | metadata | filter/re-rank by hierarchy level |
chunkset_index | metadata | stable ordering at assembly time |
| chunkset text + lineage (never the raw block array) | your doc store or .poma archive — not metadata | cheatsheet assembly under the ~40 KB cap |
Frequently asked questions
How do I get self-hosted PaddleOCR-VL results into Pinecone for RAG?
Save the raw /layout-parsing JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), embed each chunkset's to_embed, and upsert to a serverless index with compact {file_id, page, depth} metadata under a {file_id}:{chunkset_index} vector ID. page comes from the archive's chunk records (the chunkset's leaf chunk) — the SDK objects expose depth but not page. Full text stays in your doc store or the .poma archive.
Why does Pinecone's 40 KB metadata limit matter for PaddleOCR-VL's two accepted shapes?
The raw shape's parsing_res_list block array grows with page and block count; storing it as metadata for self-description hits the ~40 KB cap on exactly the dense multi-column pages PaddleOCR-VL is chosen for. Keep metadata compact instead.
What Pinecone namespace strategy fits self-hosted PaddleOCR-VL ingestion?
One namespace per tenant or corpus, with a file_id metadata filter scoping to a single document inside it — POMA writes file_id on every chunk regardless of which accepted shape produced it.
Does self-hosting PaddleOCR-VL change how I should think about Pinecone as a managed service?
It's a real tradeoff worth naming: PaddleOCR-VL keeps OCR on your infrastructure, while Pinecone is hosted. Only the layout-parsing result JSON leaves your infrastructure, first to POMA then as embeddings to Pinecone.
How do I keep PaddleOCR-VL page numbers for citations in Pinecone?
Never join markdown.text or parsing_res_list across pages before chunking. POMA maps page_index to a 1-based page number on every chunk record in the .poma archive; read the page of a chunkset's leaf chunk and put it in metadata, where numeric filters and citations can use it.
Related recipes
Same parser, different store: PaddleOCR-VL → Qdrant · PaddleOCR-VL → Weaviate · PaddleOCR-VL → Chroma
Same store, different parser: Mistral OCR → Pinecone · LlamaParse → Pinecone · AWS Textract → Pinecone
Foundations: The Optimal Chunker for PaddleOCR-VL · The Optimal Chunks for Pinecone · All pipeline recipes · RAG chunking guide