Skip to content

PaddleOCR-VL to Turbopuffer: Chunking for RAG ​

The short answer: PaddleOCR-VL gives you self-hosted, layout-labeled document parsing; Turbopuffer gives you namespace-per-tenant storage with native hybrid ANN+BM25 and server-side RRF fusion. Wired together naively — flatten the labeled blocks, split, embed, write — the pipeline underuses both, and it hits a Turbopuffer-specific snag: PaddleOCR-VL's table blocks carry raw inline HTML that a naive full-text index will happily tokenize as if it were prose. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either PaddleOCR-VL shape), emits hierarchy-preserving chunksets with tables kept structurally distinct, ready to write as rows with typed attributes.

What each end of the pipeline actually provides ​

PaddleOCR-VL runs entirely on your own infrastructure — a layout-detection model (PP-DocLayoutV3) plus a vision-language OCR model served via vLLM behind a PaddleX-compatible /layout-parsing endpoint. It returns a serving envelope with a pre-assembled markdown object, or the raw save_to_json() block list (parsing_res_list) — labeled blocks including table entries whose block_content is already inline HTML. What it doesn't provide: cross-page hierarchy, or any decision about what belongs in a full-text search field versus a filterable attribute. Details: The Optimal Chunker for PaddleOCR-VL.

Turbopuffer provides namespaces as the tenant boundary, an inline write-time schema, native hybrid search (ANN, SparseKNN, BM25) with server-side RRF fusion via multi_query(), and a storage/compute-separated architecture (SPFresh) where durable state lives in object storage. What it doesn't provide: any opinion about what a row's full_text_search-enabled attribute should contain — it ranks whatever tokens you wrote, HTML tags included. Details: Turbopuffer Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — concatenate parsing_res_list block content → RecursiveCharacterTextSplitter → embed → ns.write(...) with the whole page text marked full_text_search: true — breaks in a pair-specific way:

  • Table blocks pollute the BM25 index. PaddleOCR-VL's table-labeled blocks are already inline HTML (<table><tr><td>...). A pipeline that writes that HTML straight into a full_text_search-enabled attribute tokenizes the tags themselves alongside real content — Turbopuffer's BM25 ranking then has table, tr, and td competing with the clause numbers and defined terms a query is actually looking for, diluting exact-term precision.
  • Namespace-per-tenant gets skipped. A pipeline that writes every document into one shared namespace loses Turbopuffer's documented multi-tenancy pattern — filtering by file_id inside a shared namespace works, but it forgoes the storage isolation namespace-per-corpus was built for.
  • Overlap inflates row count. Splitter overlap embeds every boundary span twice, growing the namespace and the SPFresh index it's built over, and crowding top_k with near-duplicate rows.

Chunksets fix this at the source: PaddleOCR-VL's table blocks stay HTML in the chunk content (rendered, not tokenized as markup) while the attribute meant for full_text_search carries plain descriptive text. On a reference legal document, this discipline delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).

The pipeline, end to end ​

bash
pip install poma requests turbopuffer sentence-transformers
python
import json
import os
import requests
from poma import PrimeCut
from poma.utils import unpack_poma_archive
import turbopuffer
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
embed = lambda text: model.encode(text)

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
# keep the .poma archive: page numbers live on its chunk records, not on the SDK objects
result = poma.ingest(
    "contract.paddleocr-vl.json",  # PaddleOCR-VL shape auto-detected
    download_dir="store",
    filename="contract.poma",
)

# Chunk lookups: depth from the SDK objects, page from the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
    c["chunk_index"]: c.get("page")
    for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}

# 3. One namespace per corpus/tenant; write chunksets as rows with typed attributes.
tpuf = turbopuffer.Turbopuffer(api_key=os.environ["TURBOPUFFER_API_KEY"], region="gcp-us-central1")
ns = tpuf.namespace("contracts")

leaves = [cs.chunks[-1] for cs in result.chunksets]  # document order; last index is the leaf
ns.write(
    upsert_rows=[
        {
            "id": f"{cs.file_id}:{cs.chunkset_index}",
            "vector": embed(cs.to_embed),
            "text": cs.to_embed,
            "file_id": cs.file_id,
            "page": page_of[leaf],
            "depth": depth_of[leaf],
        }
        for cs, leaf in zip(result.chunksets, leaves)
    ],
    distance_metric="cosine_distance",
    schema={
        "text": {"type": "string", "full_text_search": True},
        "file_id": {"type": "string"},
        "page": {"type": "int"},
        "depth": {"type": "int"},
    },
)

# 4. Hybrid retrieval, fused server-side.
results = ns.multi_query(queries=[
    {"rank_by": ("vector", "ANN", embed("early termination conditions")), "top_k": 10},
    {"rank_by": ("text", "BM25", "early termination conditions"), "top_k": 10},
])

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles both PaddleOCR-VL shapes, assembles blocks by block_order rather than array position, keeps table blocks as rendered HTML rather than tokenizable markup, drops running furniture, and normalizes page_index to 1-based page numbers before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none". Merge retrieved chunksets' shared ancestor lineage into one cheatsheet before prompting.

Metadata mapping: POMA fields → Turbopuffer primitives ​

POMA chunk fieldTurbopuffer primitiveWhat it enables
to_embedvector + full_text_search: true attributedense ANN + BM25 hybrid, server-side RRF
file_idfilterable string attribute, natural namespace boundaryper-tenant/per-corpus scoping
page (on the archive's chunk records — PaddleOCR-VL's page_index, normalized to 1-based; take the chunkset's leaf chunk)filterable int attributepage-cited answers, page-range filters
depth (of the chunkset's leaf chunk, from title label levels)filterable int attributefilter/re-rank by hierarchy level
chunkset_index (document order)row id componentstable, correctly-sequenced identity
PaddleOCR-VL table blockskept as rendered HTML in text, excluded from tag-level tokenizationclean BM25 signal, tables never split

Frequently asked questions ​

How do I get self-hosted PaddleOCR-VL results into Turbopuffer for RAG? ​

Save the raw /layout-parsing response, run PrimeCut().ingest() on it, and write each chunkset as a row with a {file_id}:{chunkset_index} id, typed attributes (file_id, page, depth) and full_text_search enabled on the text. page comes from the archive's chunk records (the chunkset's leaf chunk) — the SDK objects expose depth but not page. Query with ns.multi_query() for server-side RRF fusion of dense and BM25 ranking.

Table blocks carry raw HTML as block_content. Writing that HTML straight into a full_text_search attribute tokenizes tags like table and td alongside real terms, diluting BM25 precision. Chunksets keep tables as rendered content, not tokenized markup.

Should each self-hosted PaddleOCR-VL corpus get its own Turbopuffer namespace? ​

Yes — Turbopuffer's documented pattern maps cleanly onto file_id. Self-hosted deployments especially tend to need per-tenant storage isolation, which namespace-per-corpus provides directly.

What attributes should PaddleOCR-VL chunksets carry in a Turbopuffer namespace? ​

file_id, page (normalized from page_index, read off the archive's chunk records), and depth (from title label levels) as filterable attributes, plus to_embed text with full_text_search enabled and table HTML kept out of the tokenized field.

Does chunk overlap affect a Turbopuffer namespace fed by self-hosted PaddleOCR-VL output? ​

Yes — it inflates row count and the SPFresh index built over it, crowding top_k with near-duplicates. Chunksets carry context through PaddleOCR-VL's own layout labels, so overlap is unnecessary.

Same parser, different store: PaddleOCR-VL → pgvector · PaddleOCR-VL → Milvus · PaddleOCR-VL → Vespa

Foundations: The Optimal Chunker for PaddleOCR-VL · Turbopuffer Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide