Skip to content

Turbopuffer Chunking Strategy for RAG: The Optimal Chunks for Retrieval ​

The short answer: the optimal chunks for Turbopuffer are chunksets — self-explanatory units that carry their full heading lineage — written as rows with typed attributes (file_id, chunkset_index, page, depth) inside a namespace scoped to one tenant or corpus, with full_text_search enabled on the embedded text so Turbopuffer's native hybrid ANN+BM25 ranking has both signals to fuse. Turbopuffer's object-storage-backed design means namespaces are cheap to create per tenant; what you put in each one still decides retrieval quality. POMA's PrimeCut emits exactly this row shape from any document.

Turbopuffer's namespace-per-tenant, storage-native design ​

Turbopuffer's architecture is distinctive among vector databases:

  • Namespaces — the tenant/collection boundary, created implicitly on first write, cheap enough to provision one per corpus or per customer.
  • Inline schema — attribute types (and full_text_search: true for BM25) are declared on write, not a separate DDL step.
  • Native hybrid search — rank_by supports dense ANN, SparseKNN, and BM25 on the same rows; multi_query() fuses several ranked queries with server-side RRF.
  • Storage/compute separation — durable state lives in object storage; a stateless compute layer caches hot data, so cold namespaces pay a small extra read latency and warm ones are fast.

None of this decides what a row should contain. Turbopuffer ranks whatever vectors and attributes you wrote. If a row is a context-free character-count fragment, ANN and BM25 will both faithfully rank context-free fragments — just very quickly, at very large scale.

What "optimal chunks" means for Turbopuffer, concretely ​

  1. Every row is self-explanatory. A chunkset is a root-to-leaf path through the document — Master Services Agreement → Termination clauses → Early termination → "Either party may…". Retrieved alone, it still tells the LLM exactly where it sits. See POMA chunksets.
  2. One namespace per tenant or corpus, not per document. Turbopuffer's own guidance: namespaces scope to a shared query surface. Model your document corpus as the namespace, file_id as a filterable attribute inside it — that keeps filtering fast and namespace count sane.
  3. Attributes stay under the filterable-value cap. Turbopuffer caps filterable attribute values at 4 KiB (unfiltered values can be larger, up to 8 MiB). Keep file_id, chunkset_index, page, depth as small filterable fields; the chunk text itself doesn't need to be filterable.
  4. No overlap, no near-duplicates. Overlap embeds every overlapped span twice — more rows, a larger SPFresh index, and top_k crowded with near-duplicates. Chunksets need no overlap.
  5. Hybrid by default. Enable full_text_search on the embedded text attribute so BM25 catches exact tokens dense embeddings blur, and fuse with rerank_by=("RRF",).

The pipeline: PrimeCut to Turbopuffer ​

bash
pip install turbopuffer
python
import os
from poma import PrimeCut
import turbopuffer

# 1. Chunk any document — PDF, DOCX, HTML, or a bring-your-own-OCR result JSON.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.pdf")

# 2. One namespace per corpus/tenant; write chunksets as rows with typed attributes.
tpuf = turbopuffer.Turbopuffer(api_key=os.environ["TURBOPUFFER_API_KEY"], region="gcp-us-central1")
ns = tpuf.namespace("contracts")

embed = lambda text: model.encode(text)  # your embedding model

ns.write(
    upsert_rows=[
        {
            "id": f"{cs.file_id}:{cs.chunkset_index}",
            "vector": embed(cs.to_embed),
            "text": cs.to_embed,
            "file_id": cs.file_id,
            "page": cs.chunks[0] if cs.chunks else None,
            "depth": len(cs.chunks),
        }
        for cs in result.chunksets
    ],
    distance_metric="cosine_distance",
    schema={
        "text": {"type": "string", "full_text_search": True},
        "file_id": {"type": "string"},
        "page": {"type": "int"},
        "depth": {"type": "int"},
    },
)

# 3. Hybrid retrieval, fused server-side.
results = ns.multi_query(queries=[
    {"rank_by": ("vector", "ANN", embed("early termination conditions")), "top_k": 10},
    {"rank_by": ("text", "BM25", "early termination conditions"), "top_k": 10},
])

Retrieved rows share ancestor lineage across a document — merge them into a deduplicated cheatsheet before prompting, the same way POMA's Qdrant integration does automatically. On one reference legal document this discipline answered with 337 tokens instead of 1,542 for a recursive-splitter baseline. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.

Chunk shapes in Turbopuffer, compared ​

PropertyFixed 512-token chunks + overlapSemantic chunksPOMA chunksets
Row is self-explanatory alone✗Sometimes✓ (by construction)
Namespace-per-tenant mappingManualManual✓ via file_id
Filterable attributes under 4 KiB capRisky if text is filterableRisky✓ (text unfiltered, metadata compact)
Near-duplicate rowsMany (overlap)FewNone
Hybrid ANN + BM25 with server-side RRFDIY fusionDIY fusionNative rerank_by=("RRF",)

Frequently asked questions ​

What is the optimal chunk size for Turbopuffer? ​

There's no universal token count — chunksets are self-explanatory at any size, which is why they beat fixed 512-token windows. If you must fix a size, start at 512 tokens, but the size knob cannot fix missing context.

Should each tenant get its own Turbopuffer namespace? ​

Yes — that's Turbopuffer's documented pattern, unlike stores where tenancy is a field inside one collection. Namespaces are cheap to create per tenant or corpus.

What attributes should PrimeCut chunks carry in a Turbopuffer namespace? ​

file_id, chunkset_index, page, depth as compact filterable attributes (page comes from the chunk records in the .poma archive — the chunkset’s leaf chunk — since the SDK objects expose depth but not page), plus the embedded text with full_text_search: true. Keep filterable values under the 4 KiB cap.

Yes, natively — ANN, SparseKNN, and BM25 ranking with server-side RRF fusion via multi_query() and rerank_by=("RRF",).

How does chunk overlap affect a Turbopuffer namespace? ​

It embeds every overlapped span twice, inflating row count and crowding top_k with near-duplicates. Chunksets need no overlap.

Why does a cold Turbopuffer namespace feel slower on the first query? ​

Durable state lives in object storage; a namespace without recent traffic pays a small extra read before results return. Subsequent queries against the same namespace are fast.

Retrieval with vektoria's assemble() ​

POMA's poma.vektoria package keeps the namespace content-free: Turbopuffer stores only (id, vector, {file_id, chunkset_index}), while the actual chunk content lives on a volume (a path, s3://, or gs:// URL). After your normal Turbopuffer query, hand the raw result to assemble():

python
from poma.vektoria import assemble

res = tpuf.namespace("poma").query(rank_by=("vector", "ANN", qv), top_k=10,
                                    include_attributes=["file_id", "chunkset_index"])
context = assemble(res, volume="s3://your-bucket/poma")  # -> [{"file_id", "content"}, ...]

assemble() auto-detects Turbopuffer's result shape (one of 12 supported stores) and returns deduplicated cheatsheets. The one requirement: include_attributes=["file_id", "chunkset_index"] on the query, or Turbopuffer won't return them for assemble() to work with.

Feed Turbopuffer from the parser you already run ​

Running a different store? The same chunk design applies: Azure AI Search · Chroma · Elasticsearch · FAISS · LanceDB · Milvus · MongoDB Atlas · OpenSearch · pgvector · Pinecone · Qdrant · Redis · Vespa · Weaviate — or browse all pipeline recipes.

Fundamentals: RAG chunking guide · RAG architecture guide · POMA chunksets.