Source: http://www.poma-ai.com/docs/optimal-chunks-turbopuffer

# Turbopuffer Chunking Strategy for RAG: The Optimal Chunks for Retrieval

<ByAuthor />

**The short answer:** the optimal chunks for Turbopuffer are **chunksets** — self-explanatory units that carry their full heading lineage — written as rows with typed **attributes** (`file_id`, `page`, `depth`, `chunk_index`) inside a **namespace scoped to one tenant or corpus**, with `full_text_search` enabled on the embedded text so Turbopuffer's native hybrid ANN+BM25 ranking has both signals to fuse. Turbopuffer's object-storage-backed design means namespaces are cheap to create per tenant; what you put in each one still decides retrieval quality. POMA's PrimeCut emits exactly this row shape from any document.

## Turbopuffer's namespace-per-tenant, storage-native design

Turbopuffer's architecture is distinctive among vector databases:

- **Namespaces** — the tenant/collection boundary, created implicitly on first write, cheap enough to provision one per corpus or per customer.
- **Inline schema** — attribute types (and `full_text_search: true` for BM25) are declared on `write`, not a separate DDL step.
- **Native hybrid search** — `rank_by` supports dense `ANN`, `SparseKNN`, and `BM25` on the same rows; `multi_query()` fuses several ranked queries with **server-side RRF**.
- **Storage/compute separation** — durable state lives in object storage; a stateless compute layer caches hot data, so cold namespaces pay a small extra read latency and warm ones are fast.

None of this decides what a row should contain. Turbopuffer ranks whatever vectors and attributes you wrote. If a row is a context-free character-count fragment, ANN and BM25 will both faithfully rank context-free fragments — just very quickly, at very large scale.

## What "optimal chunks" means for Turbopuffer, concretely

1. **Every row is self-explanatory.** A chunkset is a root-to-leaf path through the document — `Master Services Agreement → Termination clauses → Early termination → "Either party may…"`. Retrieved alone, it still tells the LLM exactly where it sits. See [POMA chunksets](/learn/chunking/chunksets).
2. **One namespace per tenant or corpus, not per document.** Turbopuffer's own guidance: namespaces scope to a shared query surface. Model your document corpus as the namespace, `file_id` as a filterable attribute inside it — that keeps filtering fast and namespace count sane.
3. **Attributes stay under the filterable-value cap.** Turbopuffer caps filterable attribute values at 4 KiB (unfiltered values can be larger, up to 8 MiB). Keep `file_id`, `page`, `depth`, `chunk_index` as small filterable fields; the chunk text itself doesn't need to be filterable.
4. **No overlap, no near-duplicates.** Overlap embeds every overlapped span twice — more rows, a larger SPFresh index, and `top_k` crowded with near-duplicates. Chunksets need no overlap.
5. **Hybrid by default.** Enable `full_text_search` on the embedded text attribute so BM25 catches exact tokens dense embeddings blur, and fuse with `rerank_by=("RRF",)`.

## The pipeline: PrimeCut to Turbopuffer

```bash
pip install turbopuffer
```

```python
import os
from poma import PrimeCut
import turbopuffer

# 1. Chunk any document — PDF, DOCX, HTML, or a bring-your-own-OCR result JSON.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.pdf")

# 2. One namespace per corpus/tenant; write chunksets as rows with typed attributes.
tpuf = turbopuffer.Turbopuffer(api_key=os.environ["TURBOPUFFER_API_KEY"], region="gcp-us-central1")
ns = tpuf.namespace("contracts")

embed = lambda text: model.encode(text)  # your embedding model

ns.write(
    upsert_rows=[
        {
            "id": f"{cs.file_id}:{cs.chunkset_index}",
            "vector": embed(cs.to_embed),
            "text": cs.to_embed,
            "file_id": cs.file_id,
            "page": cs.chunks[0] if cs.chunks else None,
            "depth": len(cs.chunks),
        }
        for cs in result.chunksets
    ],
    distance_metric="cosine_distance",
    schema={
        "text": {"type": "string", "full_text_search": True},
        "file_id": {"type": "string"},
        "page": {"type": "int"},
        "depth": {"type": "int"},
    },
)

# 3. Hybrid retrieval, fused server-side.
results = ns.multi_query(queries=[
    {"rank_by": ("vector", "ANN", embed("early termination conditions")), "top_k": 10},
    {"rank_by": ("text", "BM25", "early termination conditions"), "top_k": 10},
])
```

Retrieved rows share ancestor lineage across a document — merge them into a deduplicated cheatsheet before prompting, the same way POMA's Qdrant integration does automatically. On the reference legal-document benchmark this discipline answers with **337 tokens** instead of **1,542** for a recursive-splitter baseline — [methodology here](/document-ingestion-chunking-rag).

## Chunk shapes in Turbopuffer, compared

| Property | Fixed 512-token chunks + overlap | Semantic chunks | **POMA chunksets** |
| --- | --- | --- | --- |
| Row is self-explanatory alone | ✗ | Sometimes | **✓ (by construction)** |
| Namespace-per-tenant mapping | Manual | Manual | **✓ via `file_id`** |
| Filterable attributes under 4 KiB cap | Risky if text is filterable | Risky | **✓ (text unfiltered, metadata compact)** |
| Near-duplicate rows | Many (overlap) | Few | **None** |
| Hybrid ANN + BM25 with server-side RRF | DIY fusion | DIY fusion | **Native `rerank_by=("RRF",)`** |

## Frequently asked questions

### What is the optimal chunk size for Turbopuffer?

There's no universal token count — chunksets are self-explanatory at any size, which is why they beat fixed 512-token windows. If you must fix a size, start at 512 tokens, but the size knob cannot fix missing context.

### Should each tenant get its own Turbopuffer namespace?

Yes — that's Turbopuffer's documented pattern, unlike stores where tenancy is a field inside one collection. Namespaces are cheap to create per tenant or corpus.

### What attributes should PrimeCut chunks carry in a Turbopuffer namespace?

`file_id`, `page`, `depth`, `chunk_index` as compact filterable attributes, plus the embedded text with `full_text_search: true`. Keep filterable values under the 4 KiB cap.

### Does Turbopuffer support hybrid vector and keyword search?

Yes, natively — `ANN`, `SparseKNN`, and `BM25` ranking with server-side RRF fusion via `multi_query()` and `rerank_by=("RRF",)`.

### How does chunk overlap affect a Turbopuffer namespace?

It embeds every overlapped span twice, inflating row count and crowding `top_k` with near-duplicates. Chunksets need no overlap.

### Why does a cold Turbopuffer namespace feel slower on the first query?

Durable state lives in object storage; a namespace without recent traffic pays a small extra read before results return. Subsequent queries against the same namespace are fast.

## Retrieval with vektoria's assemble()

POMA's `poma.vektoria` package keeps the namespace content-free: Turbopuffer stores only `(id, vector, {file_id, chunkset_index})`, while the actual chunk content lives on a **volume** (a path, `s3://`, or `gs://` URL). After your normal Turbopuffer query, hand the raw result to `assemble()`:

```python
from poma.vektoria import assemble

res = tpuf.namespace("poma").query(rank_by=("vector", "ANN", qv), top_k=10,
                                    include_attributes=["file_id", "chunkset_index"])
context = assemble(res, volume="s3://your-bucket/poma")  # -> [{"file_id", "content"}, ...]
```

`assemble()` auto-detects Turbopuffer's result shape (one of 12 supported stores) and returns deduplicated cheatsheets. The one requirement: `include_attributes=["file_id", "chunkset_index"]` on the query, or Turbopuffer won't return them for `assemble()` to work with.

## Feed Turbopuffer from the parser you already run

- [Mistral OCR → Turbopuffer](/pipelines/mistral-ocr-to-turbopuffer)
- [LlamaParse → Turbopuffer](/pipelines/llamaparse-to-turbopuffer)
- [Azure Document Intelligence → Turbopuffer](/pipelines/azure-document-intelligence-to-turbopuffer)
- [Unstructured.io → Turbopuffer](/pipelines/unstructured-to-turbopuffer)
- [Docling → Turbopuffer](/pipelines/docling-to-turbopuffer)
- [Marker → Turbopuffer](/pipelines/marker-to-turbopuffer)
- [AWS Textract → Turbopuffer](/pipelines/textract-to-turbopuffer)
- [PaddleOCR-VL → Turbopuffer](/pipelines/paddleocr-vl-to-turbopuffer)

Running a different store? [Qdrant](/optimal-chunks-qdrant) · [Pinecone](/optimal-chunks-pinecone) · [Weaviate](/optimal-chunks-weaviate) · [pgvector](/optimal-chunks-pgvector) · [Milvus](/optimal-chunks-milvus) · [Chroma](/optimal-chunks-chroma) · [Vespa](/optimal-chunks-vespa).

Fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [RAG architecture guide](/guides/rag-architecture/) · [POMA chunksets](/learn/chunking/chunksets).