Skip to content

Docling to LanceDB: Chunking for RAG ​

The short answer: Docling gives you an explicit, typed document tree; LanceDB gives you an embedded, Arrow-native table with no server to operate. Wired together the common way — flatten to markdown, split, embed, one row per fragment — both tools end up worse than either alone, because nothing rebuilds cross-page hierarchy or keeps the table content-free. The missing link is POMA: PrimeCut().ingest() consumes the DoclingDocument tree directly, emits hierarchy-preserving chunksets, and vektoria writes them into LanceDB as content-free rows — vector plus {file_id, chunkset_index} only — while the actual text lives on a volume. Retrieval runs LanceDB's own search(), then assemble() reconstructs prompt-ready cheatsheets.

What each end of the pipeline actually provides ​

Docling (IBM's open-source converter, also served via docling-serve) returns a typed DoclingDocument: texts/tables/pictures/groups, with section_header items carrying an explicit numeric level and repeating page furniture pre-isolated into a furniture group labeled page_header/page_footer. What it doesn't provide is a retrieval unit — its own HybridChunker packs tree items into token windows sized for an embedding model, which controls length, not whether a retrieved passage carries the heading path that explains it. Details: The Optimal Chunker for Docling.

LanceDB provides an Arrow/Lance-backed table you create_table from a list of dicts — embedded in-process or object-storage-backed, no server, .where(...) for SQL-like scalar filtering alongside vector search. What it doesn't provide is any opinion about what a row should represent. It ranks whatever vector and columns you wrote. Details: LanceDB Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — doc.export_to_markdown() → RecursiveCharacterTextSplitter → one table row per fragment — breaks in a pair-specific way:

  • Docling's section_header levels are discarded at export. The tree already told you a passage lives under Termination clauses under Master Services Agreement; flattening to markdown reduces that to ## glyphs the splitter ignores, cutting wherever the character count lands.
  • Furniture Docling already isolated re-enters the text stream. The furniture group and page_header/page_footer labels exist so downstream tools skip re-detecting running noise — flattening throws that classification away, and page numbers end up embedded and indexed.
  • Overlap inflates the same Arrow table LanceDB indexes. Splitter overlap duplicates every boundary span into rows create_table writes, growing both the table on disk or object storage and the vector index built over it, with top-k crowded by near-duplicate rows instead of the passage that actually answers the query.
  • A markdown-fragment table has no clean scalar shape. Rows written from arbitrary splitter output typically carry only {text, vector}, so .where("file_id = '...'") filtering — the thing that would let you scope a query to one document cheaply — isn't available at all, and every query scans the whole table's vectors instead of a pre-filtered subset. This is specific to Docling-sourced pipelines that skip PrimeCut: the DoclingDocument tree already carries per-item structure that would populate those scalar columns for free, and flattening throws that opportunity away before the table is even created.

The pipeline, end to end ​

bash
pip install docling lancedb
python
import json
import lancedb
from docling.document_converter import DocumentConverter
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder

# 1. Your existing Docling conversion — unchanged. Save the tree as JSON.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

# 2. The missing link — raw DoclingDocument JSON in, a portable .poma archive out.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.docling.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
db = lancedb.connect("s3://your-bucket/lancedb")

# 3. Content-free ingest — the table holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
rows = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    rows.append({
        "id": r.id,
        "vector": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
    })

tbl = db.create_table("poma", data=rows, exist_ok=True)

# 4. Vector search retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
results = tbl.search(qv).where(f"file_id = '{file_id}'").limit(10).to_list()  # POMA assigns an opaque file_id, not the file name
context = assemble(results, volume=vol)  # -> [{"file_id", "content"}, ...]

records_from_archive walks the .poma archive produced by ingest — the same archive that's the volume's source of truth — and returns one Record per chunkset: a deterministic id, the to_embed text, and a payload carrying file_id, chunkset_index, and the member chunk list. No push() connector exists for LanceDB yet; ingest here is the native lancedb client, shaped exactly the way vektoria's assemble() already expects on the way back out — and unlike Redis or MongoDB Atlas, no extra query-time flag is needed for metadata to come back.

Metadata mapping: POMA fields → LanceDB primitives ​

POMA record fieldLanceDB primitiveWhat it enables
id (chunkset_uuid)id scalar columndeterministic key — safe re-ingest, no duplicate rows
text (to_embed)vector columnthe search() similarity query itself
payload["file_id"]file_id scalar column.where("file_id = '...'") scoping to one document
payload["chunkset_index"]chunkset_index scalar columnroutes a hit back to its content on the volume
chunkset contentvolume (Volume.write_doc), never the tablesmaller table; citations reconstructed at retrieval

Frequently asked questions ​

How do I get Docling output into LanceDB for RAG? ​

Export the DoclingDocument tree with export_to_dict and save it as JSON. PrimeCut().ingest() auto-detects the shape and rebuilds hierarchy into chunksets; write each one's content to a vektoria Volume, embed its to_embed text, and add a row with the vector plus file_id/chunkset_index as scalar columns. Query with search(), then assemble().

Why not just export Docling to markdown and embed it into LanceDB directly? ​

export_to_markdown() collapses explicit section_header levels and pre-isolated furniture into display text a splitter must re-guess, and its overlap writes duplicate rows into the same Arrow table LanceDB indexes — inflating both the table and the vector index with no retrieval benefit.

What LanceDB columns should Docling chunksets carry? ​

A vector column sized to your embedder, plus file_id and chunkset_index as scalar columns. Docling's hierarchy is resolved upstream by PrimeCut, so it lives in the chunkset text on the volume, not a separate table column.

Does POMA assemble need any extra flag to read LanceDB results from Docling chunksets? ​

No. LanceDB returns all columns by default from search().to_list(), so assemble() reads file_id/chunkset_index off every hit with no extra request — the one DB pair in this series with no metadata-return gotcha.

Does LanceDB support hybrid search for Docling-parsed documents? ​

Yes — create_fts_index(...) on the chunkset's to_embed text adds a lexical signal alongside .where(...) scalar filtering, useful for Docling-sourced documents where part numbers and defined terms are exact tokens dense embeddings alone tend to blur.

Same parser, different store: Docling → Chroma · Docling → pgvector · Docling → Milvus

Foundations: The Optimal Chunker for Docling · LanceDB Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide