PaddleOCR-VL to LanceDB: Chunking for RAG
The short answer: PaddleOCR-VL gives you self-hosted, layout-aware OCR — PP-DocLayoutV3 labels every region, PaddleOCR-VL reads it, all behind an HTTP endpoint you operate yourself. LanceDB gives you an embedded, Arrow-native table with no server to run. Wired together the common way — flatten, split, embed, one row per fragment — both tools end up worse than either alone, because nothing rebuilds the document's hierarchy or decides what the table's columns should be before the first row lands. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either accepted shape), emits hierarchy-preserving chunksets, and vektoria writes them into LanceDB as content-free rows — vector plus {file_id, chunkset_index} only — while the actual text lives on a volume.
What each end of the pipeline actually provides
PaddleOCR-VL runs as a self-hosted pipeline — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a /layout-parsing HTTP API you operate yourself. It returns either the PaddleX serving envelope (result.layoutParsingResults[], each with a markdown object and a prunedResult block list) or a raw save_to_json() page dict with parsing_res_list: flat blocks carrying block_label, block_content, block_id, and block_order. Empty image, figure, or chart blocks become a visible [IMG-N] marker and are counted — loss the model surfaces, never hides. What it doesn't provide: an opinion on what a retrieval unit should be. Details: The Optimal Chunker for PaddleOCR-VL.
LanceDB provides an Arrow/Lance-backed table you create_table from a list of dicts — embedded in-process or object-storage-backed, no server, .where(...) for SQL-like scalar filtering alongside vector search. What it doesn't provide is any opinion about what a row should represent, or a way to add a column after the fact without reshaping the table. It ranks whatever vector and columns you wrote. Details: LanceDB Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — call /layout-parsing, concatenate block_content sorted by block_id (or join markdown.text and skip the images side-dict entirely), run a RecursiveCharacterTextSplitter, write one {id, vector, text} row per fragment, create_table(...) on whichever document arrives first — breaks in a pair-specific way:
- Sorting by
block_idinstead ofblock_orderscrambles reading order.block_idis the model's detection order, not reading order; on a multi-column page the two diverge, and a pipeline that flattens the wrong field embeds a logically reordered document into LanceDB's vector column. - The images side-dict gets skipped, and PaddleOCR-VL's own offloaded-image signal never becomes a column. An empty image block is supposed to be a counted, visible loss — not a silent one — but a row shaped only
{id, vector, file_id}has nowhere to put that count. - LanceDB infers its Arrow schema from the first batch of rows
create_tablesees. If that decision is made without the offloaded-image column already present, adding it later isn't a metadata tweak the way it would be in a schemaless document store — it means recreating the table's schema, and that's exactly the kind of retrofit teams defer indefinitely, leaving PaddleOCR-VL's one honest signal about missing figures invisible at the table level for good.
The pipeline, end to end
pip install requests lancedbimport json
import requests
import lancedb
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. The missing link — raw result JSON in, a portable .poma archive out.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.paddleocr-vl.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
db = lancedb.connect("s3://your-bucket/lancedb")
# 3. Content-free ingest — the table holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
rows = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
rows.append({
"id": r.id,
"vector": embedder.embed([r.text])[0],
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
})
tbl = db.create_table("poma", data=rows, exist_ok=True)
# 4. Vector search retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
results = tbl.search(qv).where(f"file_id = '{file_id}'").limit(10).to_list() # POMA assigns an opaque file_id, not the file name
context = assemble(results, volume=vol) # -> [{"file_id", "content"}, ...]records_from_archive walks the .poma archive produced by ingest — the same archive that's the volume's source of truth — and returns one Record per chunkset: a deterministic id, the to_embed text, and a payload carrying file_id, chunkset_index, and the member chunk list. No push() connector exists for LanceDB yet; ingest here is the native lancedb client, shaped exactly the way assemble() already expects on the way back out. Retrieved chunksets share ancestor lineage across a document — assemble() deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answers our reference legal-document benchmark with 337 tokens instead of 1,542 for a recursive-splitter baseline. Methodology here.
Metadata mapping: POMA fields → LanceDB primitives
| POMA record field | LanceDB primitive | What it enables |
|---|---|---|
id (chunkset_uuid) | id scalar column | deterministic key — safe re-ingest, no duplicate rows |
text (to_embed) | vector column | the search() similarity query itself |
payload["file_id"] | file_id scalar column | .where("file_id = '...'") scoping to one document |
payload["chunkset_index"] | chunkset_index scalar column | routes a hit back to its content on the volume, returned by default |
payload["chunks"] (page, depth, content, image-loss count) | volume (Volume.write_doc), never a table column unless you add one at creation | full content and hierarchy reconstructed at retrieval; extra scalar columns must exist before the first create_table batch |
Frequently asked questions
How do I get self-hosted PaddleOCR-VL results into LanceDB for RAG?
Save the raw /layout-parsing JSON, run PrimeCut().ingest() on it, then pull records_from_archive(...), write each chunkset's content to a volume, embed its to_embed text, and add a row with the vector plus file_id/chunkset_index as scalar columns to a LanceDB table. Query with search(), then assemble().
Why does my LanceDB table need to decide its columns before the first PaddleOCR-VL document lands?
create_table(name, data) infers the Arrow schema from the first batch of rows. A column left out of that first batch — say, a count of images PaddleOCR-VL couldn't extract — isn't cheap to add afterward; it means recreating the table's schema, not a one-line metadata change.
What happens to images PaddleOCR-VL couldn't extract once chunks reach LanceDB?
PaddleOCR-VL marks an empty image, figure, or chart block with a visible [IMG-N] marker and counts it. That count lives in the chunk text on the volume; it only becomes a queryable LanceDB column if you add one at table-creation time.
What LanceDB columns should PaddleOCR-VL chunksets carry?
A vector column sized to your embedder, plus file_id and chunkset_index as scalar columns — the minimum the content-free contract needs. Page numbers and heading levels stay in the chunkset content on the volume unless you add extra columns for them upfront.
Can I keep documents on my own infrastructure through both OCR and LanceDB retrieval?
Yes — PaddleOCR-VL runs self-hosted behind your own endpoint, and LanceDB is embedded or object-storage-backed with no server to operate. Only the layout-parsing result JSON needs to reach POMA's API; the source document, table, and volume can all stay inside infrastructure you control.
Related recipes
Same parser, different store: PaddleOCR-VL → Qdrant · PaddleOCR-VL → MongoDB Atlas
Foundations: The Optimal Chunker for PaddleOCR-VL · LanceDB Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide · POMA chunksets