LanceDB Chunking Strategy for RAG: The Optimal Chunks for Retrieval
The short answer: the optimal chunks for LanceDB are chunksets — self-explanatory units that carry their full heading lineage — stored as rows in an Arrow-backed table with a vector column and file_id/chunkset_index/page/depth as ordinary scalar columns, filterable with a SQL-like .where(...) predicate. LanceDB's ranking is only as good as what you indexed; POMA's PrimeCut emits exactly the fields this table needs, and its vektoria package keeps the table content-free so retrieval assembles clean context from a volume instead of duplicating document bodies into local or object storage.
LanceDB: embedded, Arrow-native, object-storage-ready
LanceDB's design favors simplicity and portability over a client-server deployment model:
- Arrow/Lance columnar format — tables are ordinary columnar data; the
vectorcolumn is just one column among your scalar metadata. - Embedded or object-storage-backed — run it in-process against a local path or directly against
s3://, no server to operate. - SQL-like filtering —
.where("file_id = '...'")applies a predicate string against scalar columns alongside the vector search. - Optional full-text index —
create_fts_index(...)adds a lexical signal for hybrid queries.
None of this decides what a row should represent. LanceDB ranks whatever vector and columns you wrote. A table whose text column holds a context-free character-count fragment will rank context-free fragments — correctly, and with no server to tune.
What "optimal chunks" means for LanceDB, concretely
- One row per chunkset, self-explanatory alone. A chunkset is a root-to-leaf path —
Master Services Agreement → Termination clauses → Early termination → "Either party may…". See POMA chunksets. - Hierarchy as scalar columns.
file_id,page,depthas ordinary columns turn.where(...)into document-aware scoping without a separate metadata store. - A full-text index for hybrid.
create_fts_index(...)on the chunkset'sto_embedtext gives lexical matching alongside vector search — useful for the exact tokens embeddings blur. - No overlap. Overlap embeds every overlapped span twice, inflating table size and the vector index built over it. Chunksets need no overlap.
- Content-free table. Store
{file_id, chunkset_index}as columns and the actual content on a volume — smaller table, citations reconstructed at retrieval time.
The pipeline: PrimeCut to LanceDB
pip install lancedbimport lancedb
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
# 1. Chunk any document and keep the .poma archive — the volume's source of truth.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.pdf", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
db = lancedb.connect("s3://your-bucket/lancedb")
# 2. Content-free ingest — table holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
rows = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
rows.append({
"id": r.id,
"vector": embedder.embed([r.text])[0],
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
})
tbl = db.create_table("poma", data=rows, exist_ok=True)
# 3. Vector search retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
results = tbl.search(qv).where(f"file_id = '{file_id}'").limit(10).to_list() # POMA assigns an opaque file_id, not the file name
context = assemble(results, volume=vol) # -> [{"file_id", "content"}, ...]Retrieved chunksets share ancestor lineage across a document — assemble() deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answered one reference legal document with 337 tokens instead of 1,542 for a recursive-splitter baseline. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.
Chunk shapes in LanceDB, compared
| Property | Fixed 512-token chunks + overlap | Semantic chunks | POMA chunksets |
|---|---|---|---|
| Row is self-explanatory alone | ✗ | Sometimes | ✓ (by construction) |
| Hierarchy as scalar columns | Manual | Manual | ✓ (file_id, page, depth) |
| Redundant vector-index entries | Many (overlap) | Few | None |
| Content-free table + volume | DIY | DIY | ✓ (vektoria assemble/Volume) |
| Metadata returned on hits | Default | Default | Default — no extra flag needed |
Frequently asked questions
What is the optimal chunk size for LanceDB?
There's no universal token count — chunksets are self-explanatory at any size, which is why they beat fixed 512-token windows. Start at 512 tokens if you must fix a size, but the size knob cannot fix missing context.
How do I structure a LanceDB table for chunksets?
One row per chunkset: a vector column sized to your embedder, plus file_id, chunkset_index, page, depth as scalar columns. create_table(name, data) infers the schema from your first batch of rows.
Does LanceDB support metadata filtering and hybrid search?
Yes — .where("file_id = '...'") applies a SQL-like predicate alongside vector search, and create_fts_index(...) adds an optional full-text signal for hybrid queries.
How does POMA retrieve context from LanceDB results?
vektoria keeps the table content-free (vector + file_id/chunkset_index columns only); content lives on a volume. assemble(results, volume=VOL) auto-detects LanceDB's result shape and returns deduplicated cheatsheets — no extra flag needed.
Is LanceDB a good fit for a RAG prototype that needs to scale later?
Yes — its embedded, Arrow/Lance-backed design scales to object storage without a schema rewrite, so a table designed with the chunkset shape from day one carries into a larger deployment unchanged.
Feed LanceDB from the parser you already run
- AWS Textract → LanceDB
- Azure Document Intelligence → LanceDB
- Docling → LanceDB
- LlamaParse → LanceDB
- Marker → LanceDB
- Mistral OCR → LanceDB
- PaddleOCR-VL → LanceDB
- Unstructured.io → LanceDB
Running a different store? The same chunk design applies: Azure AI Search · Chroma · Elasticsearch · FAISS · Milvus · MongoDB Atlas · OpenSearch · pgvector · Pinecone · Qdrant · Redis · Turbopuffer · Vespa · Weaviate — or browse all pipeline recipes.
Fundamentals: RAG chunking guide · RAG architecture guide · POMA chunksets.