Docling to LanceDB: Chunking for RAG
The short answer: Docling gives you an explicit, typed document tree; LanceDB gives you an embedded, Arrow-native table with no server to operate. Wired together the common way — flatten to markdown, split, embed, one row per fragment — both tools end up worse than either alone, because nothing rebuilds cross-page hierarchy or keeps the table content-free. The missing link is POMA: PrimeCut().ingest() consumes the DoclingDocument tree directly, emits hierarchy-preserving chunksets, and vektoria writes them into LanceDB as content-free rows — vector plus {file_id, chunkset_index} only — while the actual text lives on a volume. Retrieval runs LanceDB's own search(), then assemble() reconstructs prompt-ready cheatsheets.
What each end of the pipeline actually provides
Docling (IBM's open-source converter, also served via docling-serve) returns a typed DoclingDocument: texts/tables/pictures/groups, with section_header items carrying an explicit numeric level and repeating page furniture pre-isolated into a furniture group labeled page_header/page_footer. What it doesn't provide is a retrieval unit — its own HybridChunker packs tree items into token windows sized for an embedding model, which controls length, not whether a retrieved passage carries the heading path that explains it. Details: The Optimal Chunker for Docling.
LanceDB provides an Arrow/Lance-backed table you create_table from a list of dicts — embedded in-process or object-storage-backed, no server, .where(...) for SQL-like scalar filtering alongside vector search. What it doesn't provide is any opinion about what a row should represent. It ranks whatever vector and columns you wrote. Details: LanceDB Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — doc.export_to_markdown() → RecursiveCharacterTextSplitter → one table row per fragment — breaks in a pair-specific way:
- Docling's
section_headerlevels are discarded at export. The tree already told you a passage lives under Termination clauses under Master Services Agreement; flattening to markdown reduces that to##glyphs the splitter ignores, cutting wherever the character count lands. - Furniture Docling already isolated re-enters the text stream. The
furnituregroup andpage_header/page_footerlabels exist so downstream tools skip re-detecting running noise — flattening throws that classification away, and page numbers end up embedded and indexed. - Overlap inflates the same Arrow table LanceDB indexes. Splitter overlap duplicates every boundary span into rows
create_tablewrites, growing both the table on disk or object storage and the vector index built over it, with top-k crowded by near-duplicate rows instead of the passage that actually answers the query. - A markdown-fragment table has no clean scalar shape. Rows written from arbitrary splitter output typically carry only
{text, vector}, so.where("file_id = '...'")filtering — the thing that would let you scope a query to one document cheaply — isn't available at all, and every query scans the whole table's vectors instead of a pre-filtered subset. This is specific to Docling-sourced pipelines that skip PrimeCut: the DoclingDocument tree already carries per-item structure that would populate those scalar columns for free, and flattening throws that opportunity away before the table is even created.
The pipeline, end to end
pip install docling lancedbimport json
import lancedb
from docling.document_converter import DocumentConverter
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
# 1. Your existing Docling conversion — unchanged. Save the tree as JSON.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
json.dump(doc.export_to_dict(), f)
# 2. The missing link — raw DoclingDocument JSON in, a portable .poma archive out.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.docling.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
db = lancedb.connect("s3://your-bucket/lancedb")
# 3. Content-free ingest — the table holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
rows = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
rows.append({
"id": r.id,
"vector": embedder.embed([r.text])[0],
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
})
tbl = db.create_table("poma", data=rows, exist_ok=True)
# 4. Vector search retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
results = tbl.search(qv).where(f"file_id = '{file_id}'").limit(10).to_list() # POMA assigns an opaque file_id, not the file name
context = assemble(results, volume=vol) # -> [{"file_id", "content"}, ...]records_from_archive walks the .poma archive produced by ingest — the same archive that's the volume's source of truth — and returns one Record per chunkset: a deterministic id, the to_embed text, and a payload carrying file_id, chunkset_index, and the member chunk list. No push() connector exists for LanceDB yet; ingest here is the native lancedb client, shaped exactly the way vektoria's assemble() already expects on the way back out — and unlike Redis or MongoDB Atlas, no extra query-time flag is needed for metadata to come back.
Metadata mapping: POMA fields → LanceDB primitives
| POMA record field | LanceDB primitive | What it enables |
|---|---|---|
id (chunkset_uuid) | id scalar column | deterministic key — safe re-ingest, no duplicate rows |
text (to_embed) | vector column | the search() similarity query itself |
payload["file_id"] | file_id scalar column | .where("file_id = '...'") scoping to one document |
payload["chunkset_index"] | chunkset_index scalar column | routes a hit back to its content on the volume |
| chunkset content | volume (Volume.write_doc), never the table | smaller table; citations reconstructed at retrieval |
Frequently asked questions
How do I get Docling output into LanceDB for RAG?
Export the DoclingDocument tree with export_to_dict and save it as JSON. PrimeCut().ingest() auto-detects the shape and rebuilds hierarchy into chunksets; write each one's content to a vektoria Volume, embed its to_embed text, and add a row with the vector plus file_id/chunkset_index as scalar columns. Query with search(), then assemble().
Why not just export Docling to markdown and embed it into LanceDB directly?
export_to_markdown() collapses explicit section_header levels and pre-isolated furniture into display text a splitter must re-guess, and its overlap writes duplicate rows into the same Arrow table LanceDB indexes — inflating both the table and the vector index with no retrieval benefit.
What LanceDB columns should Docling chunksets carry?
A vector column sized to your embedder, plus file_id and chunkset_index as scalar columns. Docling's hierarchy is resolved upstream by PrimeCut, so it lives in the chunkset text on the volume, not a separate table column.
Does POMA assemble need any extra flag to read LanceDB results from Docling chunksets?
No. LanceDB returns all columns by default from search().to_list(), so assemble() reads file_id/chunkset_index off every hit with no extra request — the one DB pair in this series with no metadata-return gotcha.
Does LanceDB support hybrid search for Docling-parsed documents?
Yes — create_fts_index(...) on the chunkset's to_embed text adds a lexical signal alongside .where(...) scalar filtering, useful for Docling-sourced documents where part numbers and defined terms are exact tokens dense embeddings alone tend to blur.
Related recipes
Same parser, different store: Docling → Chroma · Docling → pgvector · Docling → Milvus
Foundations: The Optimal Chunker for Docling · LanceDB Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide