Turbopuffer Chunking Strategy for RAG: The Optimal Chunks for Retrieval
The short answer: the optimal chunks for Turbopuffer are chunksets — self-explanatory units that carry their full heading lineage — written as rows with typed attributes (file_id, chunkset_index, page, depth) inside a namespace scoped to one tenant or corpus, with full_text_search enabled on the embedded text so Turbopuffer's native hybrid ANN+BM25 ranking has both signals to fuse. Turbopuffer's object-storage-backed design means namespaces are cheap to create per tenant; what you put in each one still decides retrieval quality. POMA's PrimeCut emits exactly this row shape from any document.
Turbopuffer's namespace-per-tenant, storage-native design
Turbopuffer's architecture is distinctive among vector databases:
- Namespaces — the tenant/collection boundary, created implicitly on first write, cheap enough to provision one per corpus or per customer.
- Inline schema — attribute types (and
full_text_search: truefor BM25) are declared onwrite, not a separate DDL step. - Native hybrid search —
rank_bysupports denseANN,SparseKNN, andBM25on the same rows;multi_query()fuses several ranked queries with server-side RRF. - Storage/compute separation — durable state lives in object storage; a stateless compute layer caches hot data, so cold namespaces pay a small extra read latency and warm ones are fast.
None of this decides what a row should contain. Turbopuffer ranks whatever vectors and attributes you wrote. If a row is a context-free character-count fragment, ANN and BM25 will both faithfully rank context-free fragments — just very quickly, at very large scale.
What "optimal chunks" means for Turbopuffer, concretely
- Every row is self-explanatory. A chunkset is a root-to-leaf path through the document —
Master Services Agreement → Termination clauses → Early termination → "Either party may…". Retrieved alone, it still tells the LLM exactly where it sits. See POMA chunksets. - One namespace per tenant or corpus, not per document. Turbopuffer's own guidance: namespaces scope to a shared query surface. Model your document corpus as the namespace,
file_idas a filterable attribute inside it — that keeps filtering fast and namespace count sane. - Attributes stay under the filterable-value cap. Turbopuffer caps filterable attribute values at 4 KiB (unfiltered values can be larger, up to 8 MiB). Keep
file_id,chunkset_index,page,depthas small filterable fields; the chunk text itself doesn't need to be filterable. - No overlap, no near-duplicates. Overlap embeds every overlapped span twice — more rows, a larger SPFresh index, and
top_kcrowded with near-duplicates. Chunksets need no overlap. - Hybrid by default. Enable
full_text_searchon the embedded text attribute so BM25 catches exact tokens dense embeddings blur, and fuse withrerank_by=("RRF",).
The pipeline: PrimeCut to Turbopuffer
pip install turbopufferimport os
from poma import PrimeCut
import turbopuffer
# 1. Chunk any document — PDF, DOCX, HTML, or a bring-your-own-OCR result JSON.
client = PrimeCut() # reads POMA_API_KEY
result = client.ingest("contract.pdf")
# 2. One namespace per corpus/tenant; write chunksets as rows with typed attributes.
tpuf = turbopuffer.Turbopuffer(api_key=os.environ["TURBOPUFFER_API_KEY"], region="gcp-us-central1")
ns = tpuf.namespace("contracts")
embed = lambda text: model.encode(text) # your embedding model
ns.write(
upsert_rows=[
{
"id": f"{cs.file_id}:{cs.chunkset_index}",
"vector": embed(cs.to_embed),
"text": cs.to_embed,
"file_id": cs.file_id,
"page": cs.chunks[0] if cs.chunks else None,
"depth": len(cs.chunks),
}
for cs in result.chunksets
],
distance_metric="cosine_distance",
schema={
"text": {"type": "string", "full_text_search": True},
"file_id": {"type": "string"},
"page": {"type": "int"},
"depth": {"type": "int"},
},
)
# 3. Hybrid retrieval, fused server-side.
results = ns.multi_query(queries=[
{"rank_by": ("vector", "ANN", embed("early termination conditions")), "top_k": 10},
{"rank_by": ("text", "BM25", "early termination conditions"), "top_k": 10},
])Retrieved rows share ancestor lineage across a document — merge them into a deduplicated cheatsheet before prompting, the same way POMA's Qdrant integration does automatically. On one reference legal document this discipline answered with 337 tokens instead of 1,542 for a recursive-splitter baseline. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.
Chunk shapes in Turbopuffer, compared
| Property | Fixed 512-token chunks + overlap | Semantic chunks | POMA chunksets |
|---|---|---|---|
| Row is self-explanatory alone | ✗ | Sometimes | ✓ (by construction) |
| Namespace-per-tenant mapping | Manual | Manual | ✓ via file_id |
| Filterable attributes under 4 KiB cap | Risky if text is filterable | Risky | ✓ (text unfiltered, metadata compact) |
| Near-duplicate rows | Many (overlap) | Few | None |
| Hybrid ANN + BM25 with server-side RRF | DIY fusion | DIY fusion | Native rerank_by=("RRF",) |
Frequently asked questions
What is the optimal chunk size for Turbopuffer?
There's no universal token count — chunksets are self-explanatory at any size, which is why they beat fixed 512-token windows. If you must fix a size, start at 512 tokens, but the size knob cannot fix missing context.
Should each tenant get its own Turbopuffer namespace?
Yes — that's Turbopuffer's documented pattern, unlike stores where tenancy is a field inside one collection. Namespaces are cheap to create per tenant or corpus.
What attributes should PrimeCut chunks carry in a Turbopuffer namespace?
file_id, chunkset_index, page, depth as compact filterable attributes (page comes from the chunk records in the .poma archive — the chunkset’s leaf chunk — since the SDK objects expose depth but not page), plus the embedded text with full_text_search: true. Keep filterable values under the 4 KiB cap.
Does Turbopuffer support hybrid vector and keyword search?
Yes, natively — ANN, SparseKNN, and BM25 ranking with server-side RRF fusion via multi_query() and rerank_by=("RRF",).
How does chunk overlap affect a Turbopuffer namespace?
It embeds every overlapped span twice, inflating row count and crowding top_k with near-duplicates. Chunksets need no overlap.
Why does a cold Turbopuffer namespace feel slower on the first query?
Durable state lives in object storage; a namespace without recent traffic pays a small extra read before results return. Subsequent queries against the same namespace are fast.
Retrieval with vektoria's assemble()
POMA's poma.vektoria package keeps the namespace content-free: Turbopuffer stores only (id, vector, {file_id, chunkset_index}), while the actual chunk content lives on a volume (a path, s3://, or gs:// URL). After your normal Turbopuffer query, hand the raw result to assemble():
from poma.vektoria import assemble
res = tpuf.namespace("poma").query(rank_by=("vector", "ANN", qv), top_k=10,
include_attributes=["file_id", "chunkset_index"])
context = assemble(res, volume="s3://your-bucket/poma") # -> [{"file_id", "content"}, ...]assemble() auto-detects Turbopuffer's result shape (one of 12 supported stores) and returns deduplicated cheatsheets. The one requirement: include_attributes=["file_id", "chunkset_index"] on the query, or Turbopuffer won't return them for assemble() to work with.
Feed Turbopuffer from the parser you already run
- AWS Textract → Turbopuffer
- Azure Document Intelligence → Turbopuffer
- Docling → Turbopuffer
- LlamaParse → Turbopuffer
- Marker → Turbopuffer
- Mistral OCR → Turbopuffer
- PaddleOCR-VL → Turbopuffer
- Unstructured.io → Turbopuffer
Running a different store? The same chunk design applies: Azure AI Search · Chroma · Elasticsearch · FAISS · LanceDB · Milvus · MongoDB Atlas · OpenSearch · pgvector · Pinecone · Qdrant · Redis · Vespa · Weaviate — or browse all pipeline recipes.
Fundamentals: RAG chunking guide · RAG architecture guide · POMA chunksets.