Skip to content

Marker to Turbopuffer: Chunking for RAG ​

The short answer: Marker gives you fast, local PDF parsing with real recovered structure — headings, lists, and full HTML tables inside its JSON block tree. Turbopuffer gives you a namespace-per-tenant vector store with native hybrid ANN+BM25 ranking at object-storage scale. Wired together naively, the pipeline drops every figure and risks blowing past Turbopuffer's filterable-attribute cap on wide tables before a single row is written. The missing link is POMA: PrimeCut().ingest() turns the saved Marker JSON into hierarchy-preserving chunksets, and poma.vektoria writes them to Turbopuffer as content-free rows — id, vector, and two routing attributes — while the actual chunkset text lives on a volume. Retrieval runs Turbopuffer's own query, then assemble() reconstructs prompt-ready cheatsheets.

What each end of the pipeline actually provides ​

Marker (by Datalab / VikParuchuri) runs on your own hardware and turns PDFs into clean structured output. With the JSON renderer (--output_format json) you get a Document block tree — Page children, per-block HTML, and full <table> elements with row and column spans intact. What it doesn't provide: cross-page hierarchy, retrieval units, or inline images — figure bytes live in a side-channel images dict keyed by name, with only ![](name) references left in the text. Details: The Optimal Chunker for Marker.

Turbopuffer provides namespaces (the tenant/collection boundary, created implicitly on first write), typed row attributes declared inline on write, and native hybrid search — ANN, SparseKNN, and BM25 ranking fused server-side via rerank_by=("RRF",). Storage/compute are separated: durable state lives in object storage, so cold namespaces pay a small extra read before hot data is cached. What it doesn't provide: any opinion about what a row should contain, or a document store — that's the poma.vektoria volume's job. Details: Turbopuffer Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — run Marker's default markdown output, RecursiveCharacterTextSplitter, embed, ns.write(upsert_rows=[...]) with the raw chunk text as a row attribute — breaks in ways specific to this pair:

  • Every figure vanishes before Turbopuffer sees a row. Marker's images live in the side-channel images dict; the markdown carries only ![](name) references. A markdown-only pipeline embeds those dangling references as literal text, so the chart that answers "what does the payment schedule look like" was never in the vector at all.
  • Wide tables risk the filterable-attribute cap. Marker's JSON renderer preserves full <table> elements with colspans — genuinely useful structure. Writing that raw HTML straight into a Turbopuffer attribute you intend to filter on courts the documented 4 KiB filterable-value limit; a naive splitter that cuts the table mid-row loses the span structure anyway, so neither path keeps the table usable.
  • Page numbers never make it into any filter, because Marker's markdown output has no page boundaries — only the JSON renderer's Page blocks carry them, and only if your chunker actually reads them.
  • Overlap inflates the namespace. Splitter overlap embeds every boundary span twice, growing the SPFresh index and crowding top_k with near-duplicates of the same passage.

The pipeline, end to end ​

bash
pip install turbopuffer

# Your existing Marker run — unchanged. The JSON renderer keeps the
# Document block tree that POMA auto-detects, tables and pages included.
marker_single contract.pdf --output_format json --output_dir out/
python
import os
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
import turbopuffer

# 1. Chunk the raw Marker JSON and keep the .poma archive — the volume's source of truth.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("out/contract/contract.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
tpuf = turbopuffer.Turbopuffer(api_key=os.environ["TURBOPUFFER_API_KEY"], region="gcp-us-central1")
ns = tpuf.namespace("contracts")

# 2. Content-free ingest — the namespace holds only routing attributes; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
rows = []
for r in records:
    vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
                                          "chunks": r.payload["chunks"], "text": r.text})
    rows.append({
        "id": r.id,
        "vector": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
    })
ns.write(
    upsert_rows=rows,
    distance_metric="cosine_distance",
    schema={"file_id": {"type": "string"}, "chunkset_index": {"type": "int"}},
)

# 3. Retrieval — Turbopuffer's normal ANN query, then assemble.
qv = embedder.embed(["What are the early termination conditions?"])[0]
res = ns.query(rank_by=("vector", "ANN", qv), top_k=10,
               include_attributes=["file_id", "chunkset_index"])
context = assemble(res, volume=vol)  # -> [{"file_id", "content"}, ...]
print(context[0]["content"])

records_from_archive reads the .poma archive PrimeCut just wrote and returns one Record per chunkset: .id (deterministic chunkset_uuid(file_id, chunkset_index)), .text (the to_embed string), and .payload (file_id, chunkset_index, chunks). Because the same id scheme is used on every write, re-ingesting a document overwrites the same rows rather than duplicating them. On the reference legal-document benchmark this discipline answers with 337 tokens of retrieved context instead of 1,542 for a recursive-splitter baseline — methodology here.

Metadata mapping: POMA fields → Turbopuffer primitives ​

POMA fieldWhere it livesTurbopuffer role
to_embedembedded, written as vectordense ANN ranking
file_idrow attribute (string)filterable, scopes queries to one document
chunkset_indexrow attribute (int) + volume keyforms the deterministic id; must be requested via include_attributes for assemble() to resolve content
page, depth, chunks (from Marker's Page blocks and block tree)volume only (record.payload)reconstructed into cheatsheet content, never stored as a Turbopuffer row attribute
chunkset lineagevolume documentdeduplicated and merged into one cheatsheet by assemble()

Frequently asked questions ​

How do I get Marker output into Turbopuffer for RAG? ​

Run Marker with the JSON renderer (marker_single --output_format json), hand the saved result to PrimeCut().ingest(), and keep the .poma archive. Turn each chunkset into a Record with records_from_archive, embed record.text, and ns.write() a content-free row (id, vector, file_id, chunkset_index) per record. Retrieve with ns.query() and pass the result to assemble().

Why not just split Marker's markdown and write it into a Turbopuffer namespace directly? ​

Marker's markdown has no page boundaries and leaves image bytes in a side-channel dict, so a naive splitter embeds dangling ![](name) refs and drops every figure. Writing raw chunk text (including wide HTML tables) into a filterable attribute also risks the 4 KiB filterable-value cap. The missing link keeps Marker's tables and images intact upstream and writes only compact routing attributes downstream.

What Turbopuffer attributes should Marker chunksets carry? ​

file_id and chunkset_index as typed row attributes, plus the vector. records_from_archive emits exactly these two routing fields per chunkset — the actual text and hierarchy stay on the volume.

Do Marker's extracted images and tables survive the trip to Turbopuffer? ​

Yes, upstream of Turbopuffer. PrimeCut splices Marker's side-channel images dict back into ![](name) references and describes each figure, and keeps full HTML tables intact through the chunk layer. Turbopuffer itself never stores images or tables — only a vector and two routing attributes — and the reconstructed content comes back from the volume via assemble().

Does Turbopuffer retrieval need extra fields for POMA's assemble() to work? ​

Yes — ns.query() must explicitly pass include_attributes=["file_id", "chunkset_index"], or the returned rows carry only scores and vectors and assemble() has nothing to match against the volume.

Same parser, different store: Marker → Qdrant · Marker → Pinecone · Marker → Weaviate

Foundations: The Optimal Chunker for Marker · Turbopuffer Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide