Docling to Turbopuffer: Chunking for RAG
The short answer: Docling gives you a typed DoclingDocument tree with explicit heading levels and pre-isolated page furniture; Turbopuffer gives you a namespace-per-tenant vector store with native hybrid ANN+BM25 ranking. Wired together naively, the loop that calls DocumentConverter per file tempts a namespace-per-document layout, and flattening to markdown first throws away the hierarchy Docling already recovered. The missing link is POMA: PrimeCut parses the DoclingDocument tree into chunksets, and poma.vektoria writes them as content-free Turbopuffer rows — (id, vector, {file_id, chunkset_index}) — with document content on a volume. Retrieval runs Turbopuffer's own query, then assemble() turns the hit list into prompt-ready cheatsheets.
What each end of the pipeline actually provides
Docling (IBM's open-source converter, also served via docling-serve) returns the DoclingDocument: a typed texts/tables/pictures/groups tree in which section_header items carry an explicit numeric level, tables are cell grids rather than pipe approximations, and repeating page furniture is parked in the furniture group with page_header/page_footer labels. What it doesn't provide: retrieval units, or an opinion about which vector database receives them. Details: The Optimal Chunker for Docling.
Turbopuffer provides namespaces created implicitly on first write, inline schema declaration, and native hybrid search — rank_by over dense ANN, SparseKNN, or BM25, fused server-side via multi_query() with rerank_by=("RRF",). Storage/compute separation means durable state lives in object storage, with a stateless compute layer caching hot data — cold namespaces pay a small extra read. What it doesn't provide: any opinion about what a row should contain. Details: Turbopuffer Chunking Strategy for RAG.
The naive wiring, and where it breaks
Docling's own API converts one file (or one directory) per call, and the naive integration mirrors that loop exactly: tpuf.namespace(f"doc-{filename}") per converted file. That inverts Turbopuffer's documented pattern — one namespace per tenant or corpus, not per document. Each of those per-document namespaces rarely accumulates enough steady traffic to stay warm in the compute layer's cache, so nearly every query pays a cold object-storage read, and searching across a customer's whole document set means fanning out across dozens of namespaces instead of filtering file_id inside one.
The same naive path usually skips the DoclingDocument tree entirely, calling export_to_markdown() and running a character splitter over the result — which discards the explicit section_header levels and the furniture group's pre-isolated page_header/page_footer items. So the (already too many) namespaces end up holding overlapping, unlabeled fragments: furniture text re-enters the stream, gets embedded, and — because Turbopuffer's full_text_search indexes exact tokens — a repeating footer like "Confidential — page 41" becomes a BM25-matchable row sitting next to the passage that actually answers the question.
The pipeline, end to end
pip install docling turbopuffer pomaimport os
import json
from docling.document_converter import DocumentConverter
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
import turbopuffer
# 1. Docling conversion — your existing call, unchanged.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
json.dump(doc.export_to_dict(), f)
# 2. The missing link — DoclingDocument tree in, .poma archive out.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.docling.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
tpuf = turbopuffer.Turbopuffer(api_key=os.environ["TURBOPUFFER_API_KEY"], region="gcp-us-central1")
ns = tpuf.namespace("contracts") # one namespace per corpus, not per Docling file
# 3. Content-free ingest — namespace rows hold routing metadata only.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
rows = []
for r in records:
vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
"chunks": r.payload["chunks"], "text": r.text})
rows.append({
"id": r.id,
"vector": embedder.embed([r.text])[0],
"text": r.text,
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
})
ns.write(
upsert_rows=rows,
distance_metric="cosine_distance",
schema={
"text": {"type": "string", "full_text_search": True},
"file_id": {"type": "string"},
"chunkset_index": {"type": "int"},
},
)
# 4. Retrieval — Turbopuffer's normal ANN query, then assemble.
qv = embedder.embed(["What are the early termination conditions?"])[0]
res = ns.query(rank_by=("vector", "ANN", qv), top_k=10,
include_attributes=["file_id", "chunkset_index"])
context = assemble(res, volume=vol) # -> [{"file_id", "content"}, ...]
print(context[0]["content"])Auto-detection fingerprints schema_name == "DoclingDocument" up front — a corrupted or mislabeled upload 422s immediately, never silently degrading downstream. A docling-serve envelope carrying only md_content falls back to a markdown passthrough. On our reference legal-document benchmark, this pipeline answers with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss — methodology in Document Ingestion & Chunking for RAG.
Metadata mapping: POMA fields → Turbopuffer primitives
| POMA field | Turbopuffer primitive | What it enables |
|---|---|---|
r.id (deterministic chunkset_uuid) | row id | same chunkset lands on the same row across every connector |
r.text (the to_embed text) | vector + full_text_search-enabled string attribute | dense ANN and BM25 fused via rank_by/multi_query |
file_id | string attribute (filterable) | scope queries to one document inside the shared namespace |
chunkset_index | int attribute (filterable) | must be requested via include_attributes for assemble() to resolve content |
chunks (member chunk list, including Docling section_header depth) | written to the volume document, not a namespace attribute | cheatsheet reconstruction without inflating filterable rows |
Frequently asked questions
How do I get Docling output into Turbopuffer for RAG?
Save export_to_dict() as JSON and hand it to PrimeCut, which parses the tree into chunksets and writes a .poma archive. Turn each chunkset into a row with records_from_archive() plus your embedder, upsert via ns.write, and retrieve with ns.query followed by assemble().
Should each Docling document get its own Turbopuffer namespace?
No. Docling converts one file at a time, which tempts a namespace-per-document layout, but Turbopuffer's pattern is one namespace per tenant or corpus — per-document namespaces stay cold and rarely cache-warm.
What Turbopuffer attributes should Docling chunksets carry?
file_id and chunkset_index as filterable attributes, plus the embedded text with full_text_search: true. Content itself lives on the volume, not the row.
Does Turbopuffer's hybrid search need extra fields for POMA's assemble() to work?
Yes — request include_attributes=["file_id", "chunkset_index"]. Turbopuffer doesn't return attributes by default, and without them assemble() gets nothing to resolve.
What happens to Docling's page furniture if I skip PrimeCut and write raw markdown splits to Turbopuffer?
It re-enters the text stream and gets embedded as searchable, full_text_search-indexed rows — so BM25 surfaces boilerplate like running footers next to the content that actually answers the query.
Related recipes
Same parser, different store: Docling → Vespa · Docling → Elasticsearch · Docling → Qdrant
Foundations: The Optimal Chunker for Docling · Turbopuffer Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide