Skip to content

LlamaParse to Weaviate: Chunking for RAG ​

The short answer: LlamaParse gives you excellent per-page markdown; Weaviate gives you hybrid search built in — BM25F and vector similarity fused server-side, blended by alpha. Wired together the standard LlamaIndex way, they still produce mediocre RAG, because isolated fragments starve the keyword half of exactly the section vocabulary it should be matching. The missing link is POMA: PrimeCut().ingest() consumes the saved LlamaParse result JSON (auto-detected), rebuilds the cross-page heading hierarchy into chunksets, and you insert them with the Weaviate v4 client — heading breadcrumbs feeding BM25F, self-contained text feeding the embedding, hierarchy fields as filterable properties.

What each end of the pipeline actually provides ​

LlamaParse, LlamaIndex's markdown-native parser, returns a JSON result with pages[], each carrying a page number, an md string (headings marked, tables inline), a flattened text string, and an images list — the bytes stay server-side, one /result/image/{name} fetch each. What it doesn't provide: cross-page hierarchy (each page's md is independent) or retrieval units. Details: The Optimal Chunker for LlamaParse.

Weaviate provides collections with named vectors, built-in hybrid search (BM25F + vector, weighted by alpha: 0 = pure keyword, 1 = pure vector), filterable properties, cross-references between collections, and generative/reranker modules. What it doesn't provide: any opinion about what an object should contain. Hybrid search fuses two rankings over whatever you inserted. Details: The Optimal Chunks for the Best Retrieval in Weaviate.

The naive wiring, and where it breaks ​

The canonical recipe — LlamaParse(result_type="markdown").load_data(...) → MarkdownElementNodeParser → a vector store index over Weaviate — breaks in a pair-specific way:

  • The BM25F half runs on starvation rations. This is the pair-specific waste: Weaviate is one of the few stores with keyword search built into the same query, and LlamaParse's md is full of exact, high-signal heading terms — but the node parser strips fragments loose from those headings, so the keyword index sees generic prose. A query for "termination clauses" keyword-matches nothing useful because the words "Termination clauses" lived in a heading no fragment carries.
  • Alpha becomes a rescue knob instead of a preference. Teams notice hybrid underperforming and push alpha toward pure vector search — paying for Weaviate's distinguishing feature and then tuning it off. No blend weight fixes objects that lack the terms.
  • Page and hierarchy properties stay empty. By the time nodes exist, pages[].page is gone, so Filter.by_property("page") has nothing to match and answers can't cite pages.
  • Dead image refs pollute both indexes. The saved result carries only ![](name) references — noise in the embedded text and in the BM25F term statistics.

Neither tool is at fault. LlamaParse recovered the structure; Weaviate would have indexed it twice over. The step between them threw it away.

The pipeline, end to end ​

bash
pip install llama-parse poma weaviate-client
python
import json
import os

import weaviate
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.init import Auth
from weaviate.classes.query import Filter

from llama_parse import LlamaParse
from poma import PrimeCut
from poma.utils import unpack_poma_archive

# 1. LlamaParse — your existing parse job, unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
    json.dump(json_result[0], f)

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.llamaparse.json",  # LlamaParse shape auto-detected
    download_dir="store",
    filename="contract.poma",    # keep the archive — it carries the page numbers
)

# Page numbers live on the archive's chunk records; the SDK's typed objects
# expose depth but not page. Look both up by chunk index.
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
page_of = {c["chunk_index"]: (c.get("page") if c.get("page") is not None else -1)
           for c in archive["chunks"]}  # INT properties cannot take None
depth_of = {c.chunk_index: c.depth for c in result.chunks}

# 3. Weaviate — one chunkset collection, hierarchy as filterable properties.
wv = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)  # or weaviate.connect_to_local()

chunksets = wv.collections.create(
    name="Chunkset",
    vectorizer_config=Configure.Vectorizer.none(),  # bring your own vectors
    properties=[
        Property(name="content", data_type=DataType.TEXT),  # BM25F searches this
        Property(name="file_id", data_type=DataType.TEXT),
        Property(name="page", data_type=DataType.INT),
        Property(name="depth", data_type=DataType.INT),
        Property(name="chunkset_index", data_type=DataType.INT),
    ],
)

for cs in result.chunksets:
    leaf = cs.chunks[-1]  # chunk indices in document order; the last one is the leaf
    chunksets.data.insert(
        properties={
            "content": cs.to_embed,   # heading lineage included → BM25F gets fed
            "file_id": cs.file_id,
            "page": page_of[leaf],    # LlamaParse pages[].page, via the archive's chunks
            "depth": depth_of[leaf],
            "chunkset_index": cs.chunkset_index,
        },
        vector=embed(cs.to_embed),    # your embedding model
    )

# 4. Hybrid query: BM25F + vector fused by alpha, scoped to one document.
question = "What are the early termination conditions?"
response = chunksets.query.hybrid(
    query=question,               # feeds the BM25F side
    vector=embed(question),       # feeds the vector side
    alpha=0.5,                    # 0 = pure keyword, 1 = pure vector
    limit=5,
    filters=Filter.by_property("file_id").equal(result.chunksets[0].file_id),
)
hits = [obj.properties["content"] for obj in response.objects]
# Deduplicate the shared heading lineage and merge into one cheatsheet.

POMA fingerprints the payload up front (pages[] with md + page — a corrupted or mislabeled upload 422s immediately), reads md and ignores the flattened text, neutralizes the server-side image refs and counts them in content_metadata, strips running headers and footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "llamaparse" or "none". Prefer a Weaviate vectorizer module over Vectorizer.none()? The collection design stays the same — the cluster embeds content for you.

The final merge — deduplicating the ancestors that retrieved chunksets share into one coherent block — is what POMA calls a cheatsheet, and it is where the token savings land: 337 tokens of context versus 1,542 for a recursive-splitter baseline on our reference legal document, with zero information loss — methodology here.

Metadata mapping: POMA fields → Weaviate primitives ​

POMA chunk fieldWeaviate primitiveWhat it enables
to_embedTEXT property (content) + object vectorone unit feeds BM25F and the embedding
file_idfilterable TEXT propertyscope hybrid queries to one document
page (on the archive's chunk records, derived from LlamaParse pages[].page; take the chunkset's leaf chunk)filterable INT propertypage-cited answers, page-range filters
depth (of the chunkset's leaf chunk)filterable INT propertyexclude deep appendix content, re-rank by level
chunkset_indexfilterable INT propertystable ordering at assembly time
chunkset ↔ chunk lineageoptional cross-reference to a Chunk collectionaudit/highlight workflows off the hot path

Frequently asked questions ​

How do I get LlamaParse output into Weaviate for RAG? ​

Save the raw JSON result, run PrimeCut().ingest() on it (auto-detected, md preferred, hierarchy rebuilt), then insert each chunkset with the Weaviate v4 client — to_embed as the searchable TEXT property, file_id/page/depth as filterable properties — and query with collection.query.hybrid(...).

Why does hybrid search underperform on my LlamaParse content in Weaviate? ​

The keyword half is starving: node-parser fragments lack the heading terms BM25F should match, so the fusion degrades toward pure vector search. Chunksets carry heading breadcrumbs in the searchable text, feeding both halves from one unit.

What Weaviate properties should LlamaParse chunks carry? ​

A TEXT property with the chunkset's to_embed text, plus filterable file_id, page (from pages[].page, read off the archive's chunk records — take the chunkset's leaf chunk), depth (the leaf chunk's depth), and chunkset_index. POMA records the page number on every chunk in the .poma archive; concatenate-then-split pipelines lose it.

Should I embed the md or the text field from LlamaParse into Weaviate? ​

Work from md — it carries the heading levels and inline tables; text flattens both away. But insert POMA's to_embed, not a raw field: a normalized, self-explanatory chunkset rather than a page blob or a fragment.

Should I model LlamaParse pages as cross-references in Weaviate? ​

Only for audit, highlighting, or re-chunking workflows. The hot retrieval path stays a single hybrid query on one flat collection — the chunkset's text already contains its lineage, and its page property already cites the source page.

Same parser, different store: LlamaParse → Qdrant · LlamaParse → Pinecone · LlamaParse → Chroma

Same store, different parser: Mistral OCR → Weaviate · Docling → Weaviate · Azure Document Intelligence → Weaviate

Foundations: The Optimal Chunker for LlamaParse · The Optimal Chunks for Weaviate · All pipeline recipes · RAG chunking guide