Source: http://www.poma-ai.com/docs/optimal-chunks-lancedb

# LanceDB Chunking Strategy for RAG: The Optimal Chunks for Retrieval

<ByAuthor />

**The short answer:** the optimal chunks for LanceDB are **chunksets** — self-explanatory units that carry their full heading lineage — stored as rows in an Arrow-backed table with a `vector` column and `file_id`/`chunkset_index`/`page`/`depth` as ordinary scalar columns, filterable with a SQL-like `.where(...)` predicate. LanceDB's ranking is only as good as what you indexed; POMA's PrimeCut emits exactly the fields this table needs, and its `vektoria` package keeps the table content-free so retrieval assembles clean context from a volume instead of duplicating document bodies into local or object storage.

## LanceDB: embedded, Arrow-native, object-storage-ready

LanceDB's design favors simplicity and portability over a client-server deployment model:

- **Arrow/Lance columnar format** — tables are ordinary columnar data; the `vector` column is just one column among your scalar metadata.
- **Embedded or object-storage-backed** — run it in-process against a local path or directly against `s3://`, no server to operate.
- **SQL-like filtering** — `.where("file_id = '...'")` applies a predicate string against scalar columns alongside the vector search.
- **Optional full-text index** — `create_fts_index(...)` adds a lexical signal for hybrid queries.

None of this decides what a row should represent. LanceDB ranks whatever vector and columns you wrote. A table whose text column holds a context-free character-count fragment will rank context-free fragments — correctly, and with no server to tune.

## What "optimal chunks" means for LanceDB, concretely

1. **One row per chunkset, self-explanatory alone.** A chunkset is a root-to-leaf path — `Master Services Agreement → Termination clauses → Early termination → "Either party may…"`. See [POMA chunksets](/learn/chunking/chunksets).
2. **Hierarchy as scalar columns.** `file_id`, `page`, `depth` as ordinary columns turn `.where(...)` into document-aware scoping without a separate metadata store.
3. **A full-text index for hybrid.** `create_fts_index(...)` on the chunkset's `to_embed` text gives lexical matching alongside vector search — useful for the exact tokens embeddings blur.
4. **No overlap.** Overlap embeds every overlapped span twice, inflating table size and the vector index built over it. Chunksets need no overlap.
5. **Content-free table.** Store `{file_id, chunkset_index}` as columns and the actual content on a volume — smaller table, citations reconstructed at retrieval time.

## The pipeline: PrimeCut to LanceDB

```bash
pip install lancedb
```

```python
import lancedb
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder

# 1. Chunk any document and keep the .poma archive — the volume's source of truth.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.pdf", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
db = lancedb.connect("s3://your-bucket/lancedb")

# 2. Content-free ingest — table holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
rows = []
for r in records:
    vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
                                          "chunks": r.payload["chunks"], "text": r.text})
    rows.append({
        "id": r.id,
        "vector": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
    })

tbl = db.create_table("poma", data=rows, exist_ok=True)

# 3. Vector search retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
results = tbl.search(qv).where("file_id = 'contract.pdf'").limit(10).to_list()
context = assemble(results, volume=vol)  # -> [{"file_id", "content"}, ...]
```

Retrieved chunksets share ancestor lineage across a document — `assemble()` deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answers our reference legal-document benchmark with **337 tokens** instead of **1,542** for a recursive-splitter baseline. [Methodology here](/document-ingestion-chunking-rag).

## Chunk shapes in LanceDB, compared

| Property | Fixed 512-token chunks + overlap | Semantic chunks | **POMA chunksets** |
| --- | --- | --- | --- |
| Row is self-explanatory alone | ✗ | Sometimes | **✓ (by construction)** |
| Hierarchy as scalar columns | Manual | Manual | **✓ (`file_id`, `page`, `depth`)** |
| Redundant vector-index entries | Many (overlap) | Few | **None** |
| Content-free table + volume | DIY | DIY | **✓ (`vektoria` `assemble`/`Volume`)** |
| Metadata returned on hits | Default | Default | **Default — no extra flag needed** |

## Frequently asked questions

### What is the optimal chunk size for LanceDB?

There's no universal token count — chunksets are self-explanatory at any size, which is why they beat fixed 512-token windows. Start at 512 tokens if you must fix a size, but the size knob cannot fix missing context.

### How do I structure a LanceDB table for chunksets?

One row per chunkset: a `vector` column sized to your embedder, plus `file_id`, `chunkset_index`, `page`, `depth` as scalar columns. `create_table(name, data)` infers the schema from your first batch of rows.

### Does LanceDB support metadata filtering and hybrid search?

Yes — `.where("file_id = '...'")` applies a SQL-like predicate alongside vector search, and `create_fts_index(...)` adds an optional full-text signal for hybrid queries.

### How does POMA retrieve context from LanceDB results?

`vektoria` keeps the table content-free (vector + `file_id`/`chunkset_index` columns only); content lives on a volume. `assemble(results, volume=VOL)` auto-detects LanceDB's result shape and returns deduplicated cheatsheets — no extra flag needed.

### Is LanceDB a good fit for a RAG prototype that needs to scale later?

Yes — its embedded, Arrow/Lance-backed design scales to object storage without a schema rewrite, so a table designed with the chunkset shape from day one carries into a larger deployment unchanged.

## Feed LanceDB from the parser you already run

- [Mistral OCR → LanceDB](/pipelines/mistral-ocr-to-lancedb)
- [LlamaParse → LanceDB](/pipelines/llamaparse-to-lancedb)
- [Azure Document Intelligence → LanceDB](/pipelines/azure-document-intelligence-to-lancedb)
- [Unstructured.io → LanceDB](/pipelines/unstructured-to-lancedb)
- [Docling → LanceDB](/pipelines/docling-to-lancedb)
- [Marker → LanceDB](/pipelines/marker-to-lancedb)
- [AWS Textract → LanceDB](/pipelines/textract-to-lancedb)
- [PaddleOCR-VL → LanceDB](/pipelines/paddleocr-vl-to-lancedb)

Running a different store? Browse [all 13 vector databases](/pipelines/).

Fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [RAG architecture guide](/guides/rag-architecture/) · [POMA chunksets](/learn/chunking/chunksets).