Source: http://www.poma-ai.com/docs/pipelines/paddleocr-vl-to-lancedb

# The Missing Link Between PaddleOCR-VL and Optimal Retrieval in LanceDB

<ByAuthor />

**The short answer:** PaddleOCR-VL gives you self-hosted, layout-aware OCR — PP-DocLayoutV3 labels every region, PaddleOCR-VL reads it, all behind an HTTP endpoint you operate yourself. LanceDB gives you an embedded, Arrow-native table with no server to run. Wired together the common way — flatten, split, embed, one row per fragment — both tools end up worse than either alone, because nothing rebuilds the document's hierarchy or decides what the table's columns should be before the first row lands. The missing link is POMA: `PrimeCut().ingest()` consumes the raw layout-parsing JSON (auto-detected, either accepted shape), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets), and `vektoria` writes them into LanceDB as content-free rows — vector plus `{file_id, chunkset_index}` only — while the actual text lives on a volume.

## What each end of the pipeline actually provides

**PaddleOCR-VL** runs as a self-hosted pipeline — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a `/layout-parsing` HTTP API you operate yourself. It returns either the PaddleX serving envelope (`result.layoutParsingResults[]`, each with a `markdown` object and a `prunedResult` block list) or a raw `save_to_json()` page dict with `parsing_res_list`: flat blocks carrying `block_label`, `block_content`, `block_id`, and `block_order`. Empty image, figure, or chart blocks become a visible `[IMG-N]` marker and are counted — loss the model surfaces, never hides. What it doesn't provide: an opinion on what a retrieval unit should be. Details: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl).

**LanceDB** provides an Arrow/Lance-backed table you `create_table` from a list of dicts — embedded in-process or object-storage-backed, no server, `.where(...)` for SQL-like scalar filtering alongside vector search. What it doesn't provide is any opinion about what a row should represent, or a way to add a column after the fact without reshaping the table. It ranks whatever vector and columns you wrote. Details: [LanceDB Chunking Strategy for RAG](/optimal-chunks-lancedb).

## The naive wiring, and where it breaks

The common recipe — call `/layout-parsing`, concatenate `block_content` sorted by `block_id` (or join `markdown.text` and skip the `images` side-dict entirely), run a `RecursiveCharacterTextSplitter`, write one `{id, vector, text}` row per fragment, `create_table(...)` on whichever document arrives first — breaks in a pair-specific way:

- **Sorting by `block_id` instead of `block_order` scrambles reading order.** `block_id` is the model's detection order, not reading order; on a multi-column page the two diverge, and a pipeline that flattens the wrong field embeds a logically reordered document into LanceDB's vector column.
- **The images side-dict gets skipped, and PaddleOCR-VL's own offloaded-image signal never becomes a column.** An empty image block is supposed to be a counted, visible loss — not a silent one — but a row shaped only `{id, vector, file_id}` has nowhere to put that count.
- **LanceDB infers its Arrow schema from the first batch of rows `create_table` sees.** If that decision is made without the offloaded-image column already present, adding it later isn't a metadata tweak the way it would be in a schemaless document store — it means recreating the table's schema, and that's exactly the kind of retrofit teams defer indefinitely, leaving PaddleOCR-VL's one honest signal about missing figures invisible at the table level for good.

## The pipeline, end to end

```bash
pip install requests lancedb
```

```python
import json
import requests
import lancedb
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. The missing link — raw result JSON in, a portable .poma archive out.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.paddleocr-vl.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
db = lancedb.connect("s3://your-bucket/lancedb")

# 3. Content-free ingest — the table holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
rows = []
for r in records:
    vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
                                          "chunks": r.payload["chunks"], "text": r.text})
    rows.append({
        "id": r.id,
        "vector": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
    })

tbl = db.create_table("poma", data=rows, exist_ok=True)

# 4. Vector search retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
results = tbl.search(qv).where("file_id = 'contract.pdf'").limit(10).to_list()
context = assemble(results, volume=vol)  # -> [{"file_id", "content"}, ...]
```

`records_from_archive` walks the `.poma` archive produced by ingest — the same archive that's the volume's source of truth — and returns one `Record` per chunkset: a deterministic `id`, the `to_embed` text, and a `payload` carrying `file_id`, `chunkset_index`, and the member chunk list. No `push()` connector exists for LanceDB yet; ingest here is the native `lancedb` client, shaped exactly the way `assemble()` already expects on the way back out. Retrieved chunksets share ancestor lineage across a document — `assemble()` deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answers our reference legal-document benchmark with **337 tokens** instead of **1,542** for a recursive-splitter baseline. [Methodology here](/document-ingestion-chunking-rag).

## Metadata mapping: POMA fields → LanceDB primitives

| POMA record field | LanceDB primitive | What it enables |
| --- | --- | --- |
| `id` (`chunkset_uuid`) | `id` scalar column | deterministic key — safe re-ingest, no duplicate rows |
| `text` (`to_embed`) | `vector` column | the `search()` similarity query itself |
| `payload["file_id"]` | `file_id` scalar column | `.where("file_id = '...'")` scoping to one document |
| `payload["chunkset_index"]` | `chunkset_index` scalar column | routes a hit back to its content on the volume, returned by default |
| `payload["chunks"]` (page, depth, content, image-loss count) | volume (`Volume.write_doc`), never a table column unless you add one at creation | full content and hierarchy reconstructed at retrieval; extra scalar columns must exist before the first `create_table` batch |

## Frequently asked questions

### How do I get self-hosted PaddleOCR-VL results into LanceDB for RAG?

Save the raw `/layout-parsing` JSON, run `PrimeCut().ingest()` on it, then pull `records_from_archive(...)`, write each chunkset's content to a volume, embed its `to_embed` text, and add a row with the vector plus `file_id`/`chunkset_index` as scalar columns to a LanceDB table. Query with `search()`, then `assemble()`.

### Why does my LanceDB table need to decide its columns before the first PaddleOCR-VL document lands?

`create_table(name, data)` infers the Arrow schema from the first batch of rows. A column left out of that first batch — say, a count of images PaddleOCR-VL couldn't extract — isn't cheap to add afterward; it means recreating the table's schema, not a one-line metadata change.

### What happens to images PaddleOCR-VL couldn't extract once chunks reach LanceDB?

PaddleOCR-VL marks an empty image, figure, or chart block with a visible `[IMG-N]` marker and counts it. That count lives in the chunk text on the volume; it only becomes a queryable LanceDB column if you add one at table-creation time.

### What LanceDB columns should PaddleOCR-VL chunksets carry?

A `vector` column sized to your embedder, plus `file_id` and `chunkset_index` as scalar columns — the minimum the content-free contract needs. Page numbers and heading levels stay in the chunkset content on the volume unless you add extra columns for them upfront.

### Can I keep documents on my own infrastructure through both OCR and LanceDB retrieval?

Yes — PaddleOCR-VL runs self-hosted behind your own endpoint, and LanceDB is embedded or object-storage-backed with no server to operate. Only the layout-parsing result JSON needs to reach POMA's API; the source document, table, and volume can all stay inside infrastructure you control.

## Related recipes

Same parser, different store: [PaddleOCR-VL → Qdrant](/pipelines/paddleocr-vl-to-qdrant) · [PaddleOCR-VL → MongoDB Atlas](/pipelines/paddleocr-vl-to-mongodb-atlas)

Foundations: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl) · [LanceDB Chunking Strategy for RAG](/optimal-chunks-lancedb) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/) · [POMA chunksets](/learn/chunking/chunksets)