Skip to content

PaddleOCR-VL to LanceDB: Chunking for RAG ​

The short answer: PaddleOCR-VL gives you self-hosted, layout-aware OCR — PP-DocLayoutV3 labels every region, PaddleOCR-VL reads it, all behind an HTTP endpoint you operate yourself. LanceDB gives you an embedded, Arrow-native table with no server to run. Wired together the common way — flatten, split, embed, one row per fragment — both tools end up worse than either alone, because nothing rebuilds the document's hierarchy or decides what the table's columns should be before the first row lands. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected, either accepted shape), emits hierarchy-preserving chunksets, and vektoria writes them into LanceDB as content-free rows — vector plus {file_id, chunkset_index} only — while the actual text lives on a volume.

What each end of the pipeline actually provides ​

PaddleOCR-VL runs as a self-hosted pipeline — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a /layout-parsing HTTP API you operate yourself. It returns either the PaddleX serving envelope (result.layoutParsingResults[], each with a markdown object and a prunedResult block list) or a raw save_to_json() page dict with parsing_res_list: flat blocks carrying block_label, block_content, block_id, and block_order. Empty image, figure, or chart blocks become a visible [IMG-N] marker and are counted — loss the model surfaces, never hides. What it doesn't provide: an opinion on what a retrieval unit should be. Details: The Optimal Chunker for PaddleOCR-VL.

LanceDB provides an Arrow/Lance-backed table you create_table from a list of dicts — embedded in-process or object-storage-backed, no server, .where(...) for SQL-like scalar filtering alongside vector search. What it doesn't provide is any opinion about what a row should represent, or a way to add a column after the fact without reshaping the table. It ranks whatever vector and columns you wrote. Details: LanceDB Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — call /layout-parsing, concatenate block_content sorted by block_id (or join markdown.text and skip the images side-dict entirely), run a RecursiveCharacterTextSplitter, write one {id, vector, text} row per fragment, create_table(...) on whichever document arrives first — breaks in a pair-specific way:

  • Sorting by block_id instead of block_order scrambles reading order. block_id is the model's detection order, not reading order; on a multi-column page the two diverge, and a pipeline that flattens the wrong field embeds a logically reordered document into LanceDB's vector column.
  • The images side-dict gets skipped, and PaddleOCR-VL's own offloaded-image signal never becomes a column. An empty image block is supposed to be a counted, visible loss — not a silent one — but a row shaped only {id, vector, file_id} has nowhere to put that count.
  • LanceDB infers its Arrow schema from the first batch of rows create_table sees. If that decision is made without the offloaded-image column already present, adding it later isn't a metadata tweak the way it would be in a schemaless document store — it means recreating the table's schema, and that's exactly the kind of retrofit teams defer indefinitely, leaving PaddleOCR-VL's one honest signal about missing figures invisible at the table level for good.

The pipeline, end to end ​

bash
pip install requests lancedb
python
import json
import requests
import lancedb
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. The missing link — raw result JSON in, a portable .poma archive out.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.paddleocr-vl.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
db = lancedb.connect("s3://your-bucket/lancedb")

# 3. Content-free ingest — the table holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
rows = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    rows.append({
        "id": r.id,
        "vector": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
    })

tbl = db.create_table("poma", data=rows, exist_ok=True)

# 4. Vector search retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
results = tbl.search(qv).where(f"file_id = '{file_id}'").limit(10).to_list()  # POMA assigns an opaque file_id, not the file name
context = assemble(results, volume=vol)  # -> [{"file_id", "content"}, ...]

records_from_archive walks the .poma archive produced by ingest — the same archive that's the volume's source of truth — and returns one Record per chunkset: a deterministic id, the to_embed text, and a payload carrying file_id, chunkset_index, and the member chunk list. No push() connector exists for LanceDB yet; ingest here is the native lancedb client, shaped exactly the way assemble() already expects on the way back out. Retrieved chunksets share ancestor lineage across a document — assemble() deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answers our reference legal-document benchmark with 337 tokens instead of 1,542 for a recursive-splitter baseline. Methodology here.

Metadata mapping: POMA fields → LanceDB primitives ​

POMA record fieldLanceDB primitiveWhat it enables
id (chunkset_uuid)id scalar columndeterministic key — safe re-ingest, no duplicate rows
text (to_embed)vector columnthe search() similarity query itself
payload["file_id"]file_id scalar column.where("file_id = '...'") scoping to one document
payload["chunkset_index"]chunkset_index scalar columnroutes a hit back to its content on the volume, returned by default
payload["chunks"] (page, depth, content, image-loss count)volume (Volume.write_doc), never a table column unless you add one at creationfull content and hierarchy reconstructed at retrieval; extra scalar columns must exist before the first create_table batch

Frequently asked questions ​

How do I get self-hosted PaddleOCR-VL results into LanceDB for RAG? ​

Save the raw /layout-parsing JSON, run PrimeCut().ingest() on it, then pull records_from_archive(...), write each chunkset's content to a volume, embed its to_embed text, and add a row with the vector plus file_id/chunkset_index as scalar columns to a LanceDB table. Query with search(), then assemble().

Why does my LanceDB table need to decide its columns before the first PaddleOCR-VL document lands? ​

create_table(name, data) infers the Arrow schema from the first batch of rows. A column left out of that first batch — say, a count of images PaddleOCR-VL couldn't extract — isn't cheap to add afterward; it means recreating the table's schema, not a one-line metadata change.

What happens to images PaddleOCR-VL couldn't extract once chunks reach LanceDB? ​

PaddleOCR-VL marks an empty image, figure, or chart block with a visible [IMG-N] marker and counts it. That count lives in the chunk text on the volume; it only becomes a queryable LanceDB column if you add one at table-creation time.

What LanceDB columns should PaddleOCR-VL chunksets carry? ​

A vector column sized to your embedder, plus file_id and chunkset_index as scalar columns — the minimum the content-free contract needs. Page numbers and heading levels stay in the chunkset content on the volume unless you add extra columns for them upfront.

Can I keep documents on my own infrastructure through both OCR and LanceDB retrieval? ​

Yes — PaddleOCR-VL runs self-hosted behind your own endpoint, and LanceDB is embedded or object-storage-backed with no server to operate. Only the layout-parsing result JSON needs to reach POMA's API; the source document, table, and volume can all stay inside infrastructure you control.

Same parser, different store: PaddleOCR-VL → Qdrant · PaddleOCR-VL → MongoDB Atlas

Foundations: The Optimal Chunker for PaddleOCR-VL · LanceDB Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide · POMA chunksets