PaddleOCR-VL to Vespa: Chunking for RAG
The short answer: PaddleOCR-VL gives you self-hosted, layout-labeled document parsing; Vespa gives you one system that natively fuses BM25 and ANN in a single multi-phase rank profile, at a scale proven for over a decade. Wired together naively — flatten every labeled block, split, embed, feed — the pipeline underuses both, and it hits a Vespa-specific snag: PaddleOCR-VL's own header/footer/page_number labels get discarded, so running furniture ends up indexed by bm25(text) on every document. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either PaddleOCR-VL shape), emits hierarchy-preserving chunksets with furniture already stripped, ready to feed as documents with attribute and tensor fields.
What each end of the pipeline actually provides
PaddleOCR-VL runs entirely on your own infrastructure — a layout-detection model (PP-DocLayoutV3) plus a vision-language OCR model served via vLLM behind a PaddleX-compatible /layout-parsing endpoint. It returns a serving envelope with a pre-assembled markdown object, or the raw save_to_json() block list (parsing_res_list) — labeled blocks that explicitly identify header, footer, and page_number entries as running furniture, distinct from doc_title/paragraph_title and body text. What it doesn't provide: any decision about what belongs in a BM25-ranked field versus what should be dropped before indexing. Details: The Optimal Chunker for PaddleOCR-VL.
Vespa provides tensor fields alongside scalar attributes in the same document (no separate vector-store round trip), native BM25 via index: enable-bm25 on any string field, and multi-phase ranking where one rank profile fuses ANN and lexical scores. What it doesn't provide: any opinion about what text belongs in that BM25 field — it ranks whatever you populated, boilerplate included. Details: Vespa Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — flatten markdown.text per page (or concatenate parsing_res_list block content without checking labels) → RecursiveCharacterTextSplitter → embed → app.feed_iterable(...) — breaks in a pair-specific way:
- Running furniture pollutes every document's BM25 field. PaddleOCR-VL explicitly labels
header,footer, andpage_numberblocks as furniture — that's the whole point of the label. A pipeline that flattensmarkdown.text(or ignores labels in the block-list fallback) without dropping them leaves a repeated banner like "Confidential — Internal Use" in thetextfield of every page-derived document. Vespa'sbm25(text)term-frequency scoring then rewards every single document for matching that boilerplate term, drowning out the real signal in the fused rank profile. - Hierarchy attribute fields end up empty. With titles collapsed to plain text,
file_id/depthattribute fields have nothing meaningful to filter or rank on — Vespa's structured YQL predicates lose their purpose. - Overlap duplicates entries in both indexes at once. Splitter overlap embeds every boundary span twice, so the HNSW graph and the BM25 inverted index both carry redundant entries.
Chunksets fix this at the source: furniture labels are honored and dropped before indexing, hierarchy comes from PP-DocLayoutV3's own title labels, and there's no overlap. On a reference legal document, this discipline delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install poma requests pyvespaimport json
import os
import requests
from poma import PrimeCut
from vespa.package import ApplicationPackage, Field, Schema, Document, HNSW, RankProfile
from vespa.deployment import VespaDocker
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
# Running furniture (header/footer/page_number labels) is already dropped.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest("contract.paddleocr-vl.json") # PaddleOCR-VL shape auto-detected
# depth lives on the chunk objects; a chunkset's own depth is that of its leaf chunk.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
# 3. One document type carrying both the embedding and the filterable hierarchy.
package = ApplicationPackage(
name="contracts",
schema=[Schema(
name="chunkset",
document=Document(fields=[
Field(name="file_id", type="string", indexing=["attribute"]),
Field(name="depth", type="int", indexing=["attribute"]),
Field(name="text", type="string", indexing=["index"], index="enable-bm25"),
Field(
name="embedding", type="tensor<float>(x[384])",
indexing=["attribute", "index"],
ann=HNSW(distance_metric="angular"),
),
]),
rank_profiles=[RankProfile(
name="fusion",
inputs=[("query(q)", "tensor<float>(x[384])")],
first_phase="closeness(field, embedding) * (1 + bm25(text))",
)],
)],
)
app = VespaDocker().deploy(application_package=package)
chunkset_docs = [
{"id": f"{cs.file_id}:{cs.chunkset_index}", "fields": {
# cs.chunks is in document order; the last index is the chunkset's leaf chunk
"file_id": cs.file_id, "depth": depth_of[cs.chunks[-1]], "text": cs.to_embed,
"embedding": embed(cs.to_embed), # your embedding model
}}
for cs in result.chunksets
]
app.feed_iterable(chunkset_docs, schema="chunkset", namespace="contracts")
# 4. One YQL query, both signals fused in the rank profile.
response = app.query(
yql="select * from sources * where userQuery() or "
"({targetHits:1000}nearestNeighbor(embedding,q))",
query="early termination conditions",
ranking="fusion",
body={"input.query(q)": "embed(early termination conditions)"},
)POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles both PaddleOCR-VL shapes, assembles blocks by block_order rather than array position, drops running furniture (header/footer/page_number) so it never reaches the BM25 field, and normalizes page_index to 1-based page numbers before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none". Merge retrieved chunksets' shared ancestor lineage into one cheatsheet before prompting.
Metadata mapping: POMA fields → Vespa primitives
| POMA chunk field | Vespa primitive | What it enables |
|---|---|---|
to_embed | tensor<float>(x[384]) field, HNSW-indexed | ANN matching in nearestNeighbor() |
to_embed text (furniture already dropped) | string field, index: enable-bm25 | clean lexical ranking, no boilerplate noise |
file_id | attribute field | scope YQL predicates to one document |
page (on the .poma archive's chunk records — PaddleOCR-VL's page_index, normalized to 1-based; take the chunkset's leaf chunk) | attribute field | page-cited answers, page-range predicates |
depth (of the chunkset's leaf chunk, from title label levels) | attribute field | filter/re-rank by hierarchy level |
| chunkset lineage | shared ancestor fields across documents | cheatsheet assembly client-side |
Frequently asked questions
How do I get self-hosted PaddleOCR-VL results into Vespa for RAG?
Save the raw /layout-parsing response, run PrimeCut().ingest() on it, and feed each chunkset as a document with file_id/page/depth attribute fields, a BM25-enabled text field, and a tensor field. Query with one YQL statement combining userQuery() and nearestNeighbor().
Why does PaddleOCR-VL's running-furniture labeling matter for Vespa's BM25 field?
PaddleOCR-VL labels header/footer/page_number blocks explicitly. Leaving them in the BM25 field means Vespa's bm25(text) scores every document for matching a repeated banner, drowning out real signal in the fused rank profile. Chunksets drop these labels before indexing.
How should I model PaddleOCR-VL chunks and chunksets in a Vespa schema?
One document type with file_id/page/depth attribute fields, a BM25-enabled text field built from real content only, and a tensor field with an HNSW index — vectors and scalars in the same document. depth comes from the chunkset's leaf chunk; page lives on the .poma archive's chunk records rather than on the SDK objects, so read it there for the same leaf chunk.
Should I use streaming or indexed mode for a self-hosted PaddleOCR-VL RAG corpus in Vespa?
Streaming mode's grouped IDs suit many small per-tenant partitions, common when self-hosting PaddleOCR-VL precisely for tenant isolation. Indexed mode suits a large shared corpus. Vespa advises against mixing both in one cluster.
Does Vespa's multi-phase ranking help with self-hosted PaddleOCR-VL documents specifically?
Yes — PaddleOCR-VL's layout labels already separate titles, tables, and body text, so the first-phase closeness(embedding) * (1 + bm25(text)) profile scores clean signal instead of boilerplate, with an optional global-phase re-rank on top.
Related recipes
Same parser, different store: PaddleOCR-VL → pgvector · PaddleOCR-VL → Milvus · PaddleOCR-VL → Turbopuffer
Foundations: The Optimal Chunker for PaddleOCR-VL · Vespa Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide