Skip to content

LlamaParse to Vespa: Chunking for RAG ​

The short answer: LlamaParse gives you excellent per-page markdown; Vespa gives you one system that natively fuses BM25, ANN, and arbitrary custom ranking at production scale. Wired together naively — flatten, split, embed — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy or keeps Vespa's documents content-free. The missing link is POMA: PrimeCut().ingest() consumes the raw LlamaParse JSON (auto-detected), emits hierarchy-preserving chunksets, and poma.vektoria feeds them as content-free documents — (id, embedding, {file_id, chunkset_index}) — with chunkset content on a volume, not in the schema. Retrieval runs a YQL query, then assemble() turns the hits into prompt-ready cheatsheets.

What each end of the pipeline actually provides ​

LlamaParse returns pages[], each with a page number, an md string (markdown, tables inline — POMA prefers this over text, its plain-flattened sibling), and an images list. Image bytes are not in the payload: they live server-side at LlamaParse behind a separate /result/image/{name} fetch, so a saved result JSON carries only ![](name) references. What it doesn't provide: cross-page hierarchy or retrieval units. Details: The Optimal Chunker for LlamaParse.

Vespa provides a document type per schema, with tensor fields carrying embeddings and an HNSW index living alongside ordinary scalar attribute fields in the same document — no separate vector-store round trip. What it doesn't provide: any opinion about what that document should represent, or whether the fields it fuses in a rank profile came from a hierarchy-aware chunker or a blind splitter. Details: Vespa Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — "\n".join(p["md"] for p in pages) → fixed-size splitter → embed → app.feed_iterable(...) — breaks in a pair-specific way:

  • Concatenation destroys per-chunkset ranking. Feed one oversized document instead of one per chunkset, and closeness(field, embedding) * (1 + bm25(text)) scores an undifferentiated blob — Vespa's multi-phase ranking has nothing granular to rank.
  • LlamaParse's page boundaries vanish at the join, so no Vespa attribute field can scope a query to a page, and per-document filtering is all that's left.
  • Dead image references pollute the BM25 field. LlamaParse's ![](name) refs, unneutralized, land in a string field indexed with enable-bm25 — junk tokens that can rank alongside real prose in the same query.

The pipeline, end to end ​

bash
pip install llama-parse pyvespa
python
import json
import os
from llama_parse import LlamaParse
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from vespa.package import ApplicationPackage, Field, Schema, Document, HNSW, RankProfile
from vespa.deployment import VespaDocker

# 1. LlamaParse — your existing call, unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
    json.dump(json_result[0], f)

# 2. The missing link — raw LlamaParse JSON in, a portable .poma archive out.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.llamaparse.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")

package = ApplicationPackage(
    name="contracts",
    schema=[Schema(
        name="poma",
        document=Document(fields=[
            Field(name="file_id", type="string", indexing=["attribute"]),
            Field(name="chunkset_index", type="int", indexing=["attribute"]),
            Field(
                name="embedding", type="tensor<float>(x[384])",
                indexing=["attribute", "index"],
                ann=HNSW(distance_metric="angular"),
            ),
        ]),
        rank_profiles=[RankProfile(
            name="poma_rank",
            inputs=[("query(q)", "tensor<float>(x[384])")],
            first_phase="closeness(field, embedding)",
        )],
    )],
)
app = VespaDocker().deploy(application_package=package)

# 3. Content-free ingest — document holds only routing attributes; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
docs = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    docs.append({"id": r.id, "fields": {
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
        "embedding": embedder.embed([r.text])[0],
    }})
app.feed_iterable(docs, schema="poma", namespace="contracts")

# 4. Retrieval — select the fields assemble() needs explicitly in the YQL, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
resp = app.query(
    yql="select file_id, chunkset_index from poma where "
        "{targetHits:100}nearestNeighbor(embedding, q) limit 10",
    ranking="poma_rank", body={"input.query(q)": qv},
)
context = assemble(resp, volume=vol)  # -> [{"file_id", "content"}, ...]

POMA validates the payload shape up front (fingerprinting pages[] with md + page; a corrupted or mislabeled upload 422s immediately), reads md over text, neutralizes offloaded image refs (counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking — the same treatment described in The Optimal Chunker for LlamaParse. records_from_archive then hands back one Record per chunkset, each with a deterministic id (chunkset_uuid(file_id, chunkset_index)) so the same chunkset always feeds to the same document.

Metadata mapping: POMA fields → Vespa primitives ​

POMA chunk fieldVespa primitiveWhat it enables
to_embedtensor<float>(x[N]) attribute + index fieldnearestNeighbor ranking against the query tensor
file_idstring attribute fieldYQL filtering, streaming-mode group scoping
chunkset_index (+ file_id)deterministic document id (chunkset_uuid)same chunkset always feeds to the same document, safe to re-run
chunks (member list, incl. page/depth)stored on the volume, not a Vespa fieldreconstructed into cheatsheets by assemble()
LlamaParse page (via chunk lineage)volume content, never a queried YQL fieldpage-cited answers assembled after retrieval, not queried in YQL

Frequently asked questions ​

How do I get LlamaParse results into Vespa for RAG? ​

Save the raw LlamaParse JSON, run PrimeCut().ingest() on it to get a .poma archive, then records_from_archive() + your own embedder to feed content-free documents (file_id, chunkset_index, embedding) into a Vespa schema. Retrieve with a YQL query that selects file_id, chunkset_index explicitly and pass the response to assemble().

Why not just concatenate LlamaParse's per-page markdown into one Vespa document? ​

Concatenation loses LlamaParse's page boundaries at the join, and a single oversized document defeats Vespa's per-chunkset ranking — the rank profile has one undifferentiated blob to score instead of many self-explanatory units.

What Vespa fields should LlamaParse chunksets carry? ​

file_id and chunkset_index as attribute fields, plus the embedding as a tensor field with its own HNSW index — the full content-free contract. Chunkset content, including LlamaParse's page numbers, lives on the volume instead.

Do LlamaParse's images survive the trip to Vespa? ​

Not as bytes — LlamaParse keeps images server-side, so a saved result JSON has only ![](name) references. POMA neutralizes these dead refs and counts each in content_metadata — visible, quantified loss, never silent junk ranked by BM25.

Why does assemble() return nothing even though my YQL query matched? ​

Vespa doesn't select every field by default. Name file_id and chunkset_index explicitly in the YQL select clause, or assemble() has nothing projected to fetch content for, even though the nearestNeighbor ranking ran correctly.

Same parser, different store: LlamaParse → Qdrant · LlamaParse → Milvus · LlamaParse → Chroma

Foundations: The Optimal Chunker for LlamaParse · Vespa Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide