PaddleOCR-VL to Turbopuffer: Chunking for RAG
The short answer: PaddleOCR-VL gives you self-hosted, layout-labeled document parsing; Turbopuffer gives you namespace-per-tenant storage with native hybrid ANN+BM25 and server-side RRF fusion. Wired together naively — flatten the labeled blocks, split, embed, write — the pipeline underuses both, and it hits a Turbopuffer-specific snag: PaddleOCR-VL's table blocks carry raw inline HTML that a naive full-text index will happily tokenize as if it were prose. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either PaddleOCR-VL shape), emits hierarchy-preserving chunksets with tables kept structurally distinct, ready to write as rows with typed attributes.
What each end of the pipeline actually provides
PaddleOCR-VL runs entirely on your own infrastructure — a layout-detection model (PP-DocLayoutV3) plus a vision-language OCR model served via vLLM behind a PaddleX-compatible /layout-parsing endpoint. It returns a serving envelope with a pre-assembled markdown object, or the raw save_to_json() block list (parsing_res_list) — labeled blocks including table entries whose block_content is already inline HTML. What it doesn't provide: cross-page hierarchy, or any decision about what belongs in a full-text search field versus a filterable attribute. Details: The Optimal Chunker for PaddleOCR-VL.
Turbopuffer provides namespaces as the tenant boundary, an inline write-time schema, native hybrid search (ANN, SparseKNN, BM25) with server-side RRF fusion via multi_query(), and a storage/compute-separated architecture (SPFresh) where durable state lives in object storage. What it doesn't provide: any opinion about what a row's full_text_search-enabled attribute should contain — it ranks whatever tokens you wrote, HTML tags included. Details: Turbopuffer Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — concatenate parsing_res_list block content → RecursiveCharacterTextSplitter → embed → ns.write(...) with the whole page text marked full_text_search: true — breaks in a pair-specific way:
- Table blocks pollute the BM25 index. PaddleOCR-VL's
table-labeled blocks are already inline HTML (<table><tr><td>...). A pipeline that writes that HTML straight into afull_text_search-enabled attribute tokenizes the tags themselves alongside real content — Turbopuffer's BM25 ranking then hastable,tr, andtdcompeting with the clause numbers and defined terms a query is actually looking for, diluting exact-term precision. - Namespace-per-tenant gets skipped. A pipeline that writes every document into one shared namespace loses Turbopuffer's documented multi-tenancy pattern — filtering by
file_idinside a shared namespace works, but it forgoes the storage isolation namespace-per-corpus was built for. - Overlap inflates row count. Splitter overlap embeds every boundary span twice, growing the namespace and the SPFresh index it's built over, and crowding
top_kwith near-duplicate rows.
Chunksets fix this at the source: PaddleOCR-VL's table blocks stay HTML in the chunk content (rendered, not tokenized as markup) while the attribute meant for full_text_search carries plain descriptive text. On a reference legal document, this discipline delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install poma requests turbopuffer sentence-transformersimport json
import os
import requests
from poma import PrimeCut
from poma.utils import unpack_poma_archive
import turbopuffer
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
embed = lambda text: model.encode(text)
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
# keep the .poma archive: page numbers live on its chunk records, not on the SDK objects
result = poma.ingest(
"contract.paddleocr-vl.json", # PaddleOCR-VL shape auto-detected
download_dir="store",
filename="contract.poma",
)
# Chunk lookups: depth from the SDK objects, page from the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
c["chunk_index"]: c.get("page")
for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}
# 3. One namespace per corpus/tenant; write chunksets as rows with typed attributes.
tpuf = turbopuffer.Turbopuffer(api_key=os.environ["TURBOPUFFER_API_KEY"], region="gcp-us-central1")
ns = tpuf.namespace("contracts")
leaves = [cs.chunks[-1] for cs in result.chunksets] # document order; last index is the leaf
ns.write(
upsert_rows=[
{
"id": f"{cs.file_id}:{cs.chunkset_index}",
"vector": embed(cs.to_embed),
"text": cs.to_embed,
"file_id": cs.file_id,
"page": page_of[leaf],
"depth": depth_of[leaf],
}
for cs, leaf in zip(result.chunksets, leaves)
],
distance_metric="cosine_distance",
schema={
"text": {"type": "string", "full_text_search": True},
"file_id": {"type": "string"},
"page": {"type": "int"},
"depth": {"type": "int"},
},
)
# 4. Hybrid retrieval, fused server-side.
results = ns.multi_query(queries=[
{"rank_by": ("vector", "ANN", embed("early termination conditions")), "top_k": 10},
{"rank_by": ("text", "BM25", "early termination conditions"), "top_k": 10},
])POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles both PaddleOCR-VL shapes, assembles blocks by block_order rather than array position, keeps table blocks as rendered HTML rather than tokenizable markup, drops running furniture, and normalizes page_index to 1-based page numbers before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none". Merge retrieved chunksets' shared ancestor lineage into one cheatsheet before prompting.
Metadata mapping: POMA fields → Turbopuffer primitives
| POMA chunk field | Turbopuffer primitive | What it enables |
|---|---|---|
to_embed | vector + full_text_search: true attribute | dense ANN + BM25 hybrid, server-side RRF |
file_id | filterable string attribute, natural namespace boundary | per-tenant/per-corpus scoping |
page (on the archive's chunk records — PaddleOCR-VL's page_index, normalized to 1-based; take the chunkset's leaf chunk) | filterable int attribute | page-cited answers, page-range filters |
depth (of the chunkset's leaf chunk, from title label levels) | filterable int attribute | filter/re-rank by hierarchy level |
chunkset_index (document order) | row id component | stable, correctly-sequenced identity |
PaddleOCR-VL table blocks | kept as rendered HTML in text, excluded from tag-level tokenization | clean BM25 signal, tables never split |
Frequently asked questions
How do I get self-hosted PaddleOCR-VL results into Turbopuffer for RAG?
Save the raw /layout-parsing response, run PrimeCut().ingest() on it, and write each chunkset as a row with a {file_id}:{chunkset_index} id, typed attributes (file_id, page, depth) and full_text_search enabled on the text. page comes from the archive's chunk records (the chunkset's leaf chunk) — the SDK objects expose depth but not page. Query with ns.multi_query() for server-side RRF fusion of dense and BM25 ranking.
Why does PaddleOCR-VL's inline table HTML need special handling before Turbopuffer's full_text_search?
Table blocks carry raw HTML as block_content. Writing that HTML straight into a full_text_search attribute tokenizes tags like table and td alongside real terms, diluting BM25 precision. Chunksets keep tables as rendered content, not tokenized markup.
Should each self-hosted PaddleOCR-VL corpus get its own Turbopuffer namespace?
Yes — Turbopuffer's documented pattern maps cleanly onto file_id. Self-hosted deployments especially tend to need per-tenant storage isolation, which namespace-per-corpus provides directly.
What attributes should PaddleOCR-VL chunksets carry in a Turbopuffer namespace?
file_id, page (normalized from page_index, read off the archive's chunk records), and depth (from title label levels) as filterable attributes, plus to_embed text with full_text_search enabled and table HTML kept out of the tokenized field.
Does chunk overlap affect a Turbopuffer namespace fed by self-hosted PaddleOCR-VL output?
Yes — it inflates row count and the SPFresh index built over it, crowding top_k with near-duplicates. Chunksets carry context through PaddleOCR-VL's own layout labels, so overlap is unnecessary.
Related recipes
Same parser, different store: PaddleOCR-VL → pgvector · PaddleOCR-VL → Milvus · PaddleOCR-VL → Vespa
Foundations: The Optimal Chunker for PaddleOCR-VL · Turbopuffer Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide