Skip to content

LlamaParse to OpenSearch: Chunking for RAG ​

The short answer: LlamaParse gives you excellent per-page markdown; OpenSearch gives you its own k-NN plugin plus a hybrid query type built for fusing lexical and vector search since forking from Elasticsearch in 2021. Wired together naively — flatten, split, embed — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy or keeps the index content-free. The missing link is POMA: PrimeCut().ingest() consumes the raw LlamaParse JSON (auto-detected), emits hierarchy-preserving chunksets, and poma.vektoria indexes them as content-free documents — vector plus file_id/chunkset_index only — with chunkset content on a volume, not in the document body. Retrieval runs a normal k-NN search, then assemble() turns the hits into prompt-ready cheatsheets.

What each end of the pipeline actually provides ​

LlamaParse returns pages[], each with a page number, an md string (markdown, tables inline — POMA prefers this over text, its plain-flattened sibling), and an images list. Image bytes are not in the payload: they live server-side at LlamaParse behind a separate /result/image/{name} fetch, so a saved result JSON carries only ![](name) references. What it doesn't provide: cross-page hierarchy or retrieval units. Details: The Optimal Chunker for LlamaParse.

OpenSearch provides knn_vector fields (HNSW via nmslib or faiss), native BM25 on any analyzed text field, and a dedicated hybrid query type that fuses lexical and vector sub-queries through a search pipeline's score normalization — a distinct mechanism from Elasticsearch's RRF. What it doesn't provide: any opinion about what a document should represent, or whether the text feeding that hybrid fusion is a structured chunkset or a dead image reference. Details: OpenSearch Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — "\n".join(p["md"] for p in pages) → fixed-size splitter → embed → osc.index(...) per fragment — breaks in a pair-specific way:

  • Dead image references leak into the hybrid query's lexical side. LlamaParse's ![](name) refs, unneutralized, sit in the analyzed text field. OpenSearch's hybrid query normalizes and sums the lexical and vector scores in a search pipeline, so that junk-token noise blends into the final ranking differently — and less predictably — than a pure vector search would have.
  • Splitter overlap inflates the k-NN plugin's index. Every overlapped span becomes its own document, and top_k fills with near-duplicates of the hit that actually answers the query.
  • LlamaParse's page numbers vanish at concatenation, so no keyword field can scope a query to a page, and per-document filtering is all that's left.

The pipeline, end to end ​

bash
pip install llama-parse opensearch-py
python
import json
import os
from llama_parse import LlamaParse
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from opensearchpy import OpenSearch

# 1. LlamaParse — your existing call, unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
    json.dump(json_result[0], f)

# 2. The missing link — raw LlamaParse JSON in, a portable .poma archive out.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.llamaparse.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
osc = OpenSearch(hosts=[{"host": "localhost", "port": 9200}])

osc.indices.create(index="poma", body={"settings": {"index.knn": True},  # knn_vector fields need index.knn=true, otherwise the mapping is rejected (and ignore=400 hides it)
                                       "mappings": {"properties": {
    "embedding": {"type": "knn_vector", "dimension": embedder.dims,
                  "method": {"name": "hnsw", "space_type": "cosinesimil", "engine": "faiss"}},
    "file_id": {"type": "keyword"},
    "chunkset_index": {"type": "integer"},
}}}, ignore=400)

# 3. Content-free ingest — index holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    osc.index(index="poma", id=r.id, body={
        "embedding": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
    })

# 4. Retrieval — OpenSearch returns the document by default, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
osc.indices.refresh(index="poma")  # make the just-indexed docs searchable before querying
res = osc.search(index="poma", body={"query": {"knn": {"embedding": {"vector": qv, "k": 10}}}})
context = assemble(res, volume=vol)  # -> [{"file_id", "content"}, ...]

POMA validates the payload shape up front (fingerprinting pages[] with md + page; a corrupted or mislabeled upload 422s immediately), reads md over text, neutralizes offloaded image refs (counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking — the same treatment described in The Optimal Chunker for LlamaParse. records_from_archive then hands back one Record per chunkset, each with a deterministic id (chunkset_uuid(file_id, chunkset_index)) so the same chunkset always indexes to the same document.

Metadata mapping: POMA fields → OpenSearch primitives ​

POMA chunk fieldOpenSearch primitiveWhat it enables
to_embedknn_vector fieldk-NN plugin ranking against the query embedding
file_idkeyword fieldbool.filter scoping per document
chunkset_index (+ file_id)deterministic document _id (chunkset_uuid)same chunkset always indexes to the same document, safe to re-run
chunks (member list, incl. page/depth)stored on the volume, not the document bodyreconstructed into cheatsheets by assemble()
LlamaParse page (via chunk lineage)volume content, never in the mappingpage-cited answers assembled after retrieval, not queried

Frequently asked questions ​

How do I get LlamaParse results into OpenSearch for RAG? ​

Save the raw LlamaParse JSON, run PrimeCut().ingest() on it to get a .poma archive, then records_from_archive() + your own embedder to index content-free documents (file_id, chunkset_index, embedding). Retrieve with a normal k-NN search and pass the response to assemble().

Why not just concatenate LlamaParse's markdown and bulk-index it into OpenSearch? ​

Concatenation discards LlamaParse's page numbers and heading levels before a document is ever written, and splitter overlap duplicates spans inside the k-NN plugin's index, crowding top_k with near-duplicates. OpenSearch then ranks whatever context-free fragments it was given.

What OpenSearch fields should LlamaParse chunksets carry? ​

file_id as a keyword field and chunkset_index as an integer, plus embedding as a knn_vector field — the full content-free contract. Chunkset content, including LlamaParse's page numbers, lives on the volume instead of the document body.

Do LlamaParse's images survive the trip to OpenSearch? ​

Not as bytes — LlamaParse keeps images server-side, so a saved result JSON has only ![](name) references. POMA neutralizes these dead refs and counts each in content_metadata — visible, quantified loss, never silent junk in an indexed document.

Is OpenSearch's hybrid query the same as Elasticsearch's for LlamaParse content? ​

No — both accept the same knn clause shape, so assemble() handles either result unmodified, but OpenSearch fuses lexical and vector sub-queries through its own hybrid query type and a normalizing search pipeline, distinct from Elasticsearch's RRF rank.

Same parser, different store: LlamaParse → Qdrant · LlamaParse → Milvus · LlamaParse → Chroma

Foundations: The Optimal Chunker for LlamaParse · OpenSearch Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide