Skip to content

PaddleOCR-VL to Pinecone: Chunking for RAG ​

The short answer: PaddleOCR-VL gives you self-hosted, layout-aware OCR you run behind your own HTTP endpoint; Pinecone gives you excellent serverless filtered ANN with namespaces. Wired together naively — flatten, split, embed, stuff everything into metadata for self-description — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy, and Pinecone's ~40 KB metadata cap punishes the workaround. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either accepted shape), emits hierarchy-preserving chunksets, and you upsert their embeddings with compact {file_id, page, depth} metadata — full text stays in your doc store or the .poma archive.

What each end of the pipeline actually provides ​

PaddleOCR-VL runs self-hosted — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a /layout-parsing HTTP API you operate. It returns either the PaddleX serving envelope (result.layoutParsingResults[], with a markdown object plus a prunedResult block list) or a raw save_to_json() page dict (parsing_res_list: flat blocks with block_label, block_content, block_id, block_order). What it doesn't provide: any retrieval unit, or a link between a heading on one page and its ancestor on another. Details: The Optimal Chunker for PaddleOCR-VL.

Pinecone provides serverless indexes, namespaces for hard tenant partitioning, metadata filtering ($eq, $in, $gte, …) under a ~40 KB per-vector metadata limit, and sparse-dense hybrid support. What it doesn't provide: any opinion about what a vector should represent, or a place for long text — metadata is a filter mechanism, not a document store. Details: The Optimal Chunks for the Best Retrieval in Pinecone.

The naive wiring, and where it breaks ​

The common recipe — call /layout-parsing, concatenate blocks or markdown.text across pages, split, embed, index.upsert(...) — breaks in a pair-specific way:

  • The metadata cap collides with PaddleOCR-VL's raw shape. To make a Pinecone hit self-describing without a second store, teams stuff the parsing_res_list block array — or the assembled block text plus lineage — into metadata, and hit the ~40 KB cap on exactly the long, multi-column, block-dense pages PaddleOCR-VL's layout model was chosen to handle.
  • page_index vanishes at the join, so no metadata field can cite a page and $gte/$lte page filters have nothing to act on.
  • Overlap inflates the namespace. Splitter overlap embeds every boundary span twice, so top-k results arrive as near-duplicates crowding out the passage that actually answers the question.

Chunksets fix this at the unit level: each is a self-explanatory root-to-leaf path, so the vector needs only compact filter metadata, not a smuggled block array. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).

The pipeline, end to end ​

bash
pip install requests poma pinecone sentence-transformers
python
import json
import os
import requests
from pinecone import Pinecone, ServerlessSpec
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
# keep the .poma archive: page numbers live on its chunk records, not on the SDK objects
result = poma.ingest(
    "contract.paddleocr-vl.json",  # PaddleOCR-VL shape auto-detected
    download_dir="store",
    filename="contract.poma",
)

# Chunk lookups: depth from the SDK objects, page from the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
    c["chunk_index"]: c.get("page")
    for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}

# 3. Embed chunksets; upsert with COMPACT metadata (text lives outside Pinecone).
model = SentenceTransformer("all-MiniLM-L6-v2")
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
pc.create_index(
    name="contracts", dimension=384, metric="cosine",
    spec=ServerlessSpec(cloud="aws", region="us-east-1"),
)
index = pc.Index("contracts")

file_id = result.chunks[0].file_id
store = {}  # your doc store — or resolve IDs from the .poma archive
vectors = []
for cs in result.chunksets:
    leaf = cs.chunks[-1]  # document order; the last chunk index is the leaf
    vec_id = f"{file_id}:{cs.chunkset_index}"
    store[vec_id] = cs.to_embed
    vectors.append({
        "id": vec_id,
        "values": model.encode(cs.to_embed).tolist(),
        "metadata": {"file_id": file_id, "page": page_of[leaf],
                     "depth": depth_of[leaf], "chunkset_index": cs.chunkset_index},
    })
index.upsert(vectors=vectors, namespace="acme-corp")

# 4. Filtered query, then cheatsheet assembly from the doc store.
q = model.encode("What are the early termination conditions?").tolist()
res = index.query(
    vector=q, top_k=3, namespace="acme-corp",
    filter={"file_id": {"$eq": file_id}}, include_metadata=True,
)
cheatsheet = "\n\n".join(store[m["id"]] for m in res["matches"])

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), prefers markdown.text when present and falls back to parsing_res_list sorted by block_order when it isn't, drops running furniture, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none".

Metadata mapping: POMA fields → Pinecone primitives ​

POMA chunk fieldPinecone primitiveWhat it enables
to_embeddense values (your embedding model)paraphrase retrieval
to_embed tokensoptional sparse values on the same recordexact-term hybrid retrieval
file_idmetadata, $eq filterscope queries to one document
page (on the archive's chunk records, from PaddleOCR-VL page_index, 1-based; take the chunkset's leaf chunk)metadata, $gte/$lte filterspage-cited answers, page-range scoping
depth (of the chunkset's leaf chunk)metadatafilter/re-rank by hierarchy level
chunkset_indexmetadatastable ordering at assembly time
chunkset text + lineage (never the raw block array)your doc store or .poma archive — not metadatacheatsheet assembly under the ~40 KB cap

Frequently asked questions ​

How do I get self-hosted PaddleOCR-VL results into Pinecone for RAG? ​

Save the raw /layout-parsing JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), embed each chunkset's to_embed, and upsert to a serverless index with compact {file_id, page, depth} metadata under a {file_id}:{chunkset_index} vector ID. page comes from the archive's chunk records (the chunkset's leaf chunk) — the SDK objects expose depth but not page. Full text stays in your doc store or the .poma archive.

Why does Pinecone's 40 KB metadata limit matter for PaddleOCR-VL's two accepted shapes? ​

The raw shape's parsing_res_list block array grows with page and block count; storing it as metadata for self-description hits the ~40 KB cap on exactly the dense multi-column pages PaddleOCR-VL is chosen for. Keep metadata compact instead.

What Pinecone namespace strategy fits self-hosted PaddleOCR-VL ingestion? ​

One namespace per tenant or corpus, with a file_id metadata filter scoping to a single document inside it — POMA writes file_id on every chunk regardless of which accepted shape produced it.

Does self-hosting PaddleOCR-VL change how I should think about Pinecone as a managed service? ​

It's a real tradeoff worth naming: PaddleOCR-VL keeps OCR on your infrastructure, while Pinecone is hosted. Only the layout-parsing result JSON leaves your infrastructure, first to POMA then as embeddings to Pinecone.

How do I keep PaddleOCR-VL page numbers for citations in Pinecone? ​

Never join markdown.text or parsing_res_list across pages before chunking. POMA maps page_index to a 1-based page number on every chunk record in the .poma archive; read the page of a chunkset's leaf chunk and put it in metadata, where numeric filters and citations can use it.

Same parser, different store: PaddleOCR-VL → Qdrant · PaddleOCR-VL → Weaviate · PaddleOCR-VL → Chroma

Same store, different parser: Mistral OCR → Pinecone · LlamaParse → Pinecone · AWS Textract → Pinecone

Foundations: The Optimal Chunker for PaddleOCR-VL · The Optimal Chunks for Pinecone · All pipeline recipes · RAG chunking guide