LlamaParse to Milvus: Chunking for RAG
The short answer: LlamaParse gives you excellent per-page markdown; Milvus gives you vector search that scales further than almost anything else, with scalar-field filtering, partitions, and hybrid sparse+dense retrieval. Wired together naively — concatenate, split, embed, insert — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy or maps it onto Milvus' filtering primitives. The missing link is POMA: PrimeCut().ingest() consumes the raw LlamaParse result JSON (auto-detected), emits hierarchy-preserving chunksets, and each chunkset becomes one Milvus entity with file_id, page, and depth as scalar fields — filterable, partitionable, citable.
What each end of the pipeline actually provides
LlamaParse (LlamaIndex's parser) returns pages[], each with a page number, an md string (markdown — headings marked, tables inline), a plain text flattening, and an images list whose bytes stay server-side. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for LlamaParse.
Milvus provides collections and partitions (a partition key hashes entities automatically — multi-tenancy without a collection per tenant), scalar fields whose boolean filter expressions apply during the ANN search, a spectrum of index types (HNSW memory-resident, IVF variants for smaller footprints, DiskANN for corpora that outgrow RAM), and — on recent versions — sparse vectors with built-in BM25 for hybrid search. What it doesn't provide: any opinion about what an entity should contain. Details: The Optimal Chunks for the Best Retrieval in Milvus.
The naive wiring, and where it breaks
The common recipe — "\n".join(p["md"] for p in pages) → RecursiveCharacterTextSplitter → embed → milvus.insert(...) — breaks in pair-specific ways:
- LlamaParse's page numbers vanish at the join, so there is no
pagescalar field to filter on and nofile_idper entity worth partitioning by — Milvus' filtered search and partition routing have nothing to work with. - Inline tables meet
max_length. LlamaParse renders tables inline inmd. Fixed splitters slice them mid-row; the usual "fix" — bigger chunks — produces strings that overflow themax_lengthyou declared on the VARCHAR text field, so inserts are rejected or truncated mid-table. Either way the table is gone at retrieval time. - Dead image references poison the sparse lane. The saved JSON carries only
refs (bytes live server-side at LlamaParse). Embedded, they are noise; put through BM25 hybrid, they are worse —img_p3_2.pngis a very rare token, and rare tokens score high, so figure filenames outrank real content in the sparse ranking. - Overlap inflates the entity count, which inflates HNSW build time, IVF training, or DiskANN's disk footprint — and fills top-k with near-identical hits.
The pipeline, end to end
pip install llama-parse poma pymilvus sentence-transformersimport json
import os
from llama_parse import LlamaParse
from pymilvus import DataType, MilvusClient
from sentence_transformers import SentenceTransformer
from poma import PrimeCut
from poma.utils import unpack_poma_archive
# 1. LlamaParse — your existing call, unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
json.dump(json_result[0], f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.llamaparse.json", # LlamaParse shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive — it carries the page numbers
)
# Page numbers live on the archive's chunk records; the SDK's typed objects
# expose depth but not page. Look both up by chunk index.
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
page_of = {c["chunk_index"]: (c.get("page") if c.get("page") is not None else -1)
for c in archive["chunks"]} # INT64 fields cannot take NULL
depth_of = {c.chunk_index: c.depth for c in result.chunks}
# 3. Milvus — a chunkset collection with hierarchy scalar fields.
milvus = MilvusClient(uri=os.environ["MILVUS_URI"], token=os.environ["MILVUS_TOKEN"])
schema = MilvusClient.create_schema(auto_id=True)
schema.add_field("pk", DataType.INT64, is_primary=True)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=384)
schema.add_field("text", DataType.VARCHAR, max_length=65535)
schema.add_field("file_id", DataType.VARCHAR, max_length=128, is_partition_key=True)
schema.add_field("page", DataType.INT64)
schema.add_field("depth", DataType.INT64)
schema.add_field("chunkset_index", DataType.INT64)
index_params = milvus.prepare_index_params()
index_params.add_index(field_name="vector", index_type="HNSW", metric_type="COSINE")
milvus.create_collection("chunksets", schema=schema, index_params=index_params)
# 4. Embed each chunkset's to_embed text and insert.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2") # 384-dim
milvus.insert(
collection_name="chunksets",
data=[
{
"vector": model.encode(cs.to_embed).tolist(),
"text": cs.to_embed,
"file_id": cs.file_id,
# cs.chunks holds chunk indices in document order; the last is the leaf.
"page": page_of[cs.chunks[-1]],
"depth": depth_of[cs.chunks[-1]],
"chunkset_index": cs.chunkset_index,
}
for cs in result.chunksets
],
)
# 5. Filtered ANN search — scoped to one document, pages citable.
hits = milvus.search(
collection_name="chunksets",
data=[model.encode("What are the early termination conditions?").tolist()],
limit=5,
filter='file_id == "contract" and depth <= 3',
output_fields=["text", "page", "depth", "chunkset_index"],
)POMA validates the payload up front (a corrupted or mislabeled upload 422s immediately), reads md over text so no structure is lost, neutralizes the dead  image references and counts them in content_metadata, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "llamaparse" or "none". After retrieval, merge hits instead of concatenating: chunksets from the same section share ancestor breadcrumbs, and deduplicating that lineage assembles one prompt-ready cheatsheet — 337 tokens of context versus 1,542 for a recursive-splitter baseline on our reference legal-document benchmark, zero information loss (methodology).
Metadata mapping: POMA fields → Milvus primitives
| POMA chunk field | Milvus primitive | What it enables |
|---|---|---|
to_embed | FLOAT_VECTOR field (HNSW/IVF/DiskANN); optional sparse BM25 vector on recent versions | paraphrase + exact-term hybrid retrieval |
file_id | VARCHAR scalar field, partition key | partition routing + per-document filter expressions |
page (on the archive's chunk records, derived from LlamaParse pages[].page; take the chunkset's leaf chunk) | INT64 scalar field | page-cited answers, page >= 40-style range filters |
depth (of the chunkset's leaf chunk) | INT64 scalar field | filter/re-rank by hierarchy level (depth <= 2) |
chunkset_index | INT64 scalar field | stable ordering at assembly time |
| chunkset lineage | breadcrumbs inside the text VARCHAR field | client-side cheatsheet assembly without a second store |
Frequently asked questions
How do I get LlamaParse results into Milvus for RAG?
Save the raw JSON result, run PrimeCut().ingest() on it (auto-detected via the pages[] + md + page fingerprint, hierarchy rebuilt), then insert one entity per chunkset with MilvusClient: embedded to_embed in a FLOAT_VECTOR field, hierarchy in scalar fields.
What Milvus schema fits LlamaParse-parsed documents?
Auto-id primary key, FLOAT_VECTOR for the embedding, VARCHAR for the chunkset text, and scalar fields file_id (partition key), page (from the archive's chunk records — take the chunkset's leaf chunk), depth (the leaf chunk's depth), chunkset_index. Add a sparse vector field for BM25 hybrid on recent Milvus versions.
Do LlamaParse's inline markdown tables fit in a Milvus VARCHAR field?
Naively, often not: splitters slice tables mid-row, and whole-page chunks with large tables overflow the declared max_length — rejected inserts or mid-table truncation. POMA keeps tables as intact chunk units; declare a generous max_length and whole tables stay retrievable.
Should I use Milvus hybrid BM25 search for LlamaParse content?
Yes, where your version supports it — exact tokens like clause numbers and defined terms need the sparse lane. But neutralize LlamaParse's dead  refs first (POMA does): image filenames are rare tokens, and rare tokens dominate BM25 scores.
How do I scope Milvus search to one LlamaParse-parsed document?
Filter on the scalar field: file_id == "contract" applies during the ANN search, and as partition key file_id also routes the query to the relevant partitions. This only works because ingest preserved file_id on every chunkset and page on every chunk record in the .poma archive.
Related recipes
Same parser, different store: LlamaParse → Qdrant · LlamaParse → pgvector · LlamaParse → Chroma
Same store, different parser: Mistral OCR → Milvus · Unstructured → Milvus · Marker → Milvus
Foundations: The Optimal Chunker for LlamaParse · The Optimal Chunks for Milvus · All pipeline recipes · RAG chunking guide