Marker to Vespa: Chunking for RAG
The short answer: Marker gives you fast, local PDF parsing with a real Document block tree — per-page blocks, explicit types, and full HTML tables with spans intact. Vespa gives you one system that natively fuses vector search, BM25, and arbitrary ranking in a single query, with vectors as ordinary tensor fields alongside scalar attributes. Wired together naively, Marker's raw block HTML lands in a Vespa field with no tokenizer awareness of tags or dangling image refs, polluting whatever signal you build on it. The missing link is POMA: PrimeCut().ingest() turns the saved Marker JSON into hierarchy-preserving chunksets, and poma.vektoria feeds them into Vespa as content-free documents — file_id, chunkset_index, and an embedding tensor — while the actual chunkset text lives on a volume. Retrieval runs a YQL query, then assemble() reconstructs prompt-ready cheatsheets.
What each end of the pipeline actually provides
Marker (by Datalab / VikParuchuri) runs on your own hardware and turns PDFs into clean structured output. With the JSON renderer (--output_format json) you get a Document block tree — Page children, per-block HTML, and full <table> elements with row and column spans intact. What it doesn't provide: cross-page hierarchy, retrieval units, or inline images — figure bytes live in a side-channel images dict keyed by name, with only  references left in the text. Details: The Optimal Chunker for Marker.
Vespa stores vectors as ordinary tensor fields inside the same document type as scalar attributes — no separate vector-store object. Its distinctive strengths: native BM25 on any string field, multi-phase rank profiles that fuse ANN and lexical scores in one pass, and tiered execution modes (indexed for large shared corpora, streaming for many small per-tenant partitions). What it doesn't provide: any interpretation of what a document field should hold, or HTML-aware tokenization — a string field indexes whatever text you fed it, tags and dangling references included. Details: Vespa Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — flatten Marker's markdown, split, embed, app.feed_iterable() a schema whose fields hold the raw text — breaks in ways specific to this pair:
- Every figure vanishes before Vespa sees a field. Marker's images live in the side-channel
imagesdict; the markdown carries onlyreferences. Fed into any Vespa field as-is, those references become just another string token matching nothing — Vespa's tokenizer has no notion that a reference is supposed to point at bytes. - Raw block HTML pollutes text fields. The JSON renderer's tables arrive as full
<table>elements with colspans — genuinely useful structure Marker recovered. If that markup is dumped straight into astringfield without conversion, Vespa indexes literal tag text (<td colspan="3">) as ordinary tokens; a query for the values inside the table ranks on tag noise rather than cell content. - Vespa's schema is fixed at deploy time. Tensor dimensions and field types are declared in the
ApplicationPackage, not inferred per document — feeding a raw, unshaped block stream means guessing field types before you've seen a well-formed document, and redeploying the schema when the shape changes. - Page boundaries never make it into a filterable field, because Marker's markdown output alone drops them — only the JSON renderer's Page blocks carry them, and only if the ingest path reads them.
The pipeline, end to end
pip install pyvespa
# Your existing Marker run — unchanged. The JSON renderer keeps the
# Document block tree that POMA auto-detects, tables and pages included.
marker_single contract.pdf --output_format json --output_dir out/from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from vespa.package import ApplicationPackage, Field, Schema, Document, HNSW, RankProfile
from vespa.deployment import VespaDocker
# 1. Chunk the raw Marker JSON and keep the .poma archive — the volume's source of truth.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("out/contract/contract.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
package = ApplicationPackage(
name="contracts",
schema=[Schema(
name="chunkset",
document=Document(fields=[
Field(name="file_id", type="string", indexing=["attribute"]),
Field(name="chunkset_index", type="int", indexing=["attribute"]),
Field(
name="embedding", type="tensor<float>(x[384])",
indexing=["attribute", "index"],
ann=HNSW(distance_metric="angular"),
),
]),
rank_profiles=[RankProfile(
name="ann",
inputs=[("query(q)", "tensor<float>(x[384])")],
first_phase="closeness(field, embedding)",
)],
)],
)
app = VespaDocker().deploy(application_package=package)
# 2. Content-free feed — the document holds only routing attributes; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
docs = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
docs.append({"id": r.id, "fields": {
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
"embedding": embedder.embed([r.text])[0],
}})
app.feed_iterable(docs, schema="chunkset", namespace="contracts")
# 3. Retrieval — select the routing fields explicitly in the YQL, then assemble.
qv = embedder.embed(["What are the early termination conditions?"])[0]
resp = app.query(
yql="select file_id, chunkset_index from sources * where "
"{targetHits:100}nearestNeighbor(embedding, q) limit 10",
ranking="ann",
body={"input.query(q)": qv},
)
context = assemble(resp, volume=vol) # -> [{"file_id", "content"}, ...]records_from_archive reads the .poma archive PrimeCut just wrote and returns one Record per chunkset: .id (deterministic chunkset_uuid(file_id, chunkset_index)), .text (the to_embed string used to build the embedding), and .payload (file_id, chunkset_index, chunks). The same id scheme means re-ingesting a document updates the same Vespa documents rather than duplicating them. On the reference legal-document benchmark this discipline answers with 337 tokens of retrieved context instead of 1,542 for a recursive-splitter baseline — methodology here.
Metadata mapping: POMA fields → Vespa primitives
| POMA field | Where it lives | Vespa role |
|---|---|---|
to_embed | embedded, written as the embedding field | tensor<float>(x[N]) with an HNSW ann index |
file_id | document attribute field | filterable via YQL predicates; must be selected explicitly for assemble() |
chunkset_index | document attribute field + volume key | forms the deterministic id; also selected explicitly in the YQL |
page, depth, chunks (from Marker's Page blocks and block tree) | volume only (record.payload) | reconstructed into cheatsheet content, never stored as a Vespa document field |
| chunkset lineage | volume document | deduplicated and merged into one cheatsheet by assemble() |
Frequently asked questions
How do I get Marker output into Vespa for RAG?
Run Marker with the JSON renderer (marker_single --output_format json), hand the saved result to PrimeCut().ingest(), and keep the .poma archive. Turn each chunkset into a Record with records_from_archive, embed record.text into the tensor field, and app.feed_iterable() a content-free document (file_id, chunkset_index, embedding) per record. Retrieve with a YQL nearestNeighbor query and pass the response to assemble().
Why not just split Marker's markdown and feed it into a Vespa schema directly?
Marker's markdown drops page boundaries and leaves image bytes in a side-channel dict, and its JSON renderer's tables arrive as full HTML with no tokenizer awareness in Vespa. Feeding that raw block HTML into a string field indexes tag text as tokens instead of table values, and dangling  refs become unmatched noise. The missing link converts blocks to clean text and keeps the fed document content-free.
How should Marker chunksets be modeled in a Vespa document type?
One document type with file_id and chunkset_index as attribute fields and a tensor<float>(x[N]) field with an HNSW index for the embedding. The document holds only the vector and two routing attributes; the chunkset's text and hierarchy live on a volume.
Do Marker's tables and images survive the trip to Vespa?
Yes, upstream of Vespa. PrimeCut splices Marker's side-channel images dict back into its references and describes each figure, and keeps full HTML tables intact through the chunk layer. Vespa itself never stores images or tables under this pattern — only the embedding and two routing attributes — and the reconstructed content comes back from the volume via assemble().
Does Vespa retrieval need special handling for POMA's assemble() to work?
Yes — the YQL query must explicitly select file_id, chunkset_index alongside the nearestNeighbor match, or the response carries only scores and assemble() has nothing to resolve against the volume.
Related recipes
Same parser, different store: Marker → Qdrant · Marker → Weaviate · Marker → pgvector
Foundations: The Optimal Chunker for Marker · Vespa Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide