Skip to content

LlamaParse to Turbopuffer: Chunking for RAG ​

The short answer: LlamaParse gives you excellent per-page markdown; Turbopuffer gives you namespace-per-tenant storage with native hybrid ANN+BM25 ranking at object-storage scale. Wired together naively — flatten, split, embed — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy or keeps the namespace content-free. The missing link is POMA: PrimeCut().ingest() consumes the raw LlamaParse JSON (auto-detected), emits hierarchy-preserving chunksets, and poma.vektoria writes them as content-free rows — (id, vector, {file_id, chunkset_index}) — with chunkset content on a volume, not in the namespace. Retrieval runs Turbopuffer's own query, then assemble() turns the hits into prompt-ready cheatsheets.

What each end of the pipeline actually provides ​

LlamaParse returns pages[], each with a page number, an md string (markdown, tables inline — POMA prefers this over text, its plain-flattened sibling), and an images list. Image bytes are not in the payload: they live server-side at LlamaParse behind a separate /result/image/{name} fetch, so a saved result JSON carries only ![](name) references. What it doesn't provide: cross-page hierarchy or retrieval units. Details: The Optimal Chunker for LlamaParse.

Turbopuffer provides namespaces (the tenant/corpus boundary, created implicitly on first write), typed attributes declared inline on write, and native rank_by hybrid search — ANN, SparseKNN, and BM25 fused server-side via rerank_by=("RRF",). What it doesn't provide: any opinion about what a row should contain, or what it means for a document's structure to survive the trip from parser to namespace. Details: Turbopuffer Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — "\n".join(p["md"] for p in pages) → fixed-size splitter → embed → ns.write(...) — breaks in a pair-specific way:

  • LlamaParse's own JSON literally has a field named text sitting next to md. Naive wiring maps that key straight onto a Turbopuffer attribute of the same name, because the names already match — silently choosing the plain-flattened rendering over the structure-carrying md field, with no chunker in between to notice.
  • Splitter overlap duplicates spans as separate rows. Every overlapped span becomes its own row in the namespace, inflating the object-storage footprint the SPFresh index is built over and crowding top_k with near-duplicates of the one row that actually answers the query.
  • Dead image references get embedded as text. LlamaParse's ![](name) refs, unneutralized, become part of whatever gets embedded — junk tokens that participate in BM25 ranking if full_text_search is enabled on that attribute.

The pipeline, end to end ​

bash
pip install llama-parse turbopuffer
python
import json
import os
from llama_parse import LlamaParse
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
import turbopuffer

# 1. LlamaParse — your existing call, unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
    json.dump(json_result[0], f)

# 2. The missing link — raw LlamaParse JSON in, a portable .poma archive out.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.llamaparse.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
tpuf = turbopuffer.Turbopuffer(api_key=os.environ["TURBOPUFFER_API_KEY"], region="gcp-us-central1")
ns = tpuf.namespace("contracts")

# 3. Content-free ingest — namespace holds only routing attributes; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
rows = []
for r in records:
    vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
                                          "chunks": r.payload["chunks"], "text": r.text})
    rows.append({
        "id": r.id,
        "vector": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
    })
ns.write(upsert_rows=rows, distance_metric="cosine_distance", schema={
    "file_id": {"type": "string"},
    "chunkset_index": {"type": "uint"},
})

# 4. Retrieval — request the attributes assemble() needs, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
res = ns.query(rank_by=("vector", "ANN", qv), top_k=10,
               include_attributes=["file_id", "chunkset_index"])
context = assemble(res, volume=vol)  # -> [{"file_id", "content"}, ...]

POMA validates the payload shape up front (fingerprinting pages[] with md + page; a corrupted or mislabeled upload 422s immediately), reads md over text, neutralizes offloaded image refs (counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking — the same treatment described in The Optimal Chunker for LlamaParse. records_from_archive then hands back one Record per chunkset, each with a deterministic id (chunkset_uuid(file_id, chunkset_index)) so the same chunkset always lands on the same row.

Metadata mapping: POMA fields → Turbopuffer primitives ​

POMA chunk fieldTurbopuffer primitiveWhat it enables
to_embedvector on the rowANN ranking against the query embedding
file_idtyped string attribute (filterable)per-document scoping, one namespace per tenant/corpus
chunkset_index (+ file_id)deterministic row id (chunkset_uuid)same chunkset always upserts to the same row, safe to re-run
chunks (member list, incl. page/depth)stored on the volume, not a Turbopuffer attributereconstructed into cheatsheets by assemble()
LlamaParse page (via chunk lineage)volume content, never filterable in the namespacepage-cited answers surface at assembly time, not query time

Frequently asked questions ​

How do I get LlamaParse results into Turbopuffer for RAG? ​

Save the raw LlamaParse JSON, run PrimeCut().ingest() on it to get a .poma archive, then records_from_archive() + your own embedder to upsert content-free rows (id, vector, file_id, chunkset_index) into a Turbopuffer namespace. Retrieve with ns.query(..., include_attributes=["file_id", "chunkset_index"]) and pass the result to assemble().

Why not just split LlamaParse's markdown and write it into Turbopuffer directly? ​

Splitting discards LlamaParse's page numbers and heading levels before a row is ever written, and overlap duplicates spans as near-identical rows that crowd top_k. Turbopuffer then ranks whatever context-free fragments it was given — the pipeline underuses both tools.

What Turbopuffer attributes should LlamaParse chunksets carry? ​

file_id and chunkset_index as typed, filterable attributes — the full content-free contract. Chunkset content, including LlamaParse's page numbers, lives on the volume instead, reconstructed by assemble() at retrieval time.

Do LlamaParse's images survive the trip to Turbopuffer? ​

Not as bytes — LlamaParse keeps images server-side, so a saved result JSON has only ![](name) references. POMA neutralizes these dead refs and counts each in content_metadata — visible, quantified loss, never silent junk in a row.

Why does a Turbopuffer query return nothing even though the ANN search matched? ​

Turbopuffer doesn't return attributes by default. Pass include_attributes=["file_id", "chunkset_index"] at query time, or assemble() has nothing to fetch content for and returns an empty list against a non-empty result.

Same parser, different store: LlamaParse → Qdrant · LlamaParse → Pinecone · LlamaParse → Weaviate

Foundations: The Optimal Chunker for LlamaParse · Turbopuffer Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide