LlamaParse to Chroma: Chunking for RAG
The short answer: LlamaParse gives you excellent per-page markdown; Chroma gives you the fastest path from zero to a working retrieval prototype — embedded, local-first, batteries included. Wired together naively — one page in, one document out — they produce a prototype that lies to you, because page-sized embeddings blur sections and the chunking decision you make now is the one production inherits. The missing link is POMA: PrimeCut().ingest() consumes the raw LlamaParse result JSON (auto-detected), emits hierarchy-preserving chunksets, and those land in Chroma as small, self-explanatory documents with where-filterable hierarchy metadata — while the .poma archive keeps the whole thing portable to any production store.
What each end of the pipeline actually provides
LlamaParse (LlamaIndex's parser) returns pages[], each with a page number, an md string (markdown — headings marked, tables inline), a plain text flattening, and an images list whose bytes stay server-side. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for LlamaParse.
Chroma provides a PersistentClient that runs embedded or client/server with the same API, collections with default or bring-your-own embedding functions, where filters over scalar metadata, and where_document filters over content. What it doesn't provide: any opinion about what a document should contain — and no escape hatch from a bad chunking decision once your evaluations are built on it. Details: The Optimal Chunks for the Best Retrieval in Chroma.
The naive wiring, and where it breaks
LlamaParse's output shape makes the naive wiring seductively easy — pages[].md maps one-to-one onto collection.add, one document per page. That is exactly the trap:
- Page-sized embeddings blur. A page mixing the end of one section and the start of the next becomes a single vector that represents neither well.
n_results=5returns five walls of text, and sections spanning page breaks are severed mid-thought. - Dead image references pollute both search lanes. The saved JSON carries only
refs (bytes live server-side at LlamaParse). Added naively, they sit inside your documents — embedded as noise, and matchable bywhere_document={"$contains": ...}, which can hitimg_p3_2.pnginstead of content. - No metadata, no filters. Whole-page adds rarely carry
file_idordepth, sowherefilters — Chroma's main retrieval control — have nothing to grip. - The prototype's chunking sticks. This is the pair's defining failure: Chroma makes starting so easy that the page-per-document decision hardens into your evaluation baseline. When production lands on another store, you re-chunk and re-evaluate everything — unless the chunks were portable from day one.
The pipeline, end to end
pip install llama-parse poma chromadbimport json
import os
import chromadb
from llama_parse import LlamaParse
from poma import PrimeCut
from poma.utils import unpack_poma_archive
# 1. LlamaParse — your existing call, unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
json.dump(json_result[0], f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.llamaparse.json", # LlamaParse shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive — it carries the page numbers
)
# Page numbers live on the archive's chunk records; the SDK's typed objects
# expose depth but not page. Look both up by chunk index.
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
page_of = {c["chunk_index"]: (c.get("page") if c.get("page") is not None else -1)
for c in archive["chunks"]} # Chroma metadata values cannot be None
depth_of = {c.chunk_index: c.depth for c in result.chunks}
# 3. Chroma — durable local collection, default embedding function (or BYO).
chroma = chromadb.PersistentClient(path="./chroma")
collection = chroma.get_or_create_collection("contracts")
# cs.chunks holds chunk indices in document order; the last one is the leaf.
collection.add(
documents=[cs.to_embed for cs in result.chunksets],
metadatas=[
{
"file_id": cs.file_id,
"page": page_of[cs.chunks[-1]],
"depth": depth_of[cs.chunks[-1]],
"chunkset_index": cs.chunkset_index,
}
for cs in result.chunksets
],
ids=[f"{cs.file_id}:{cs.chunkset_index}" for cs in result.chunksets],
)
# 4. Query — metadata and content filters compose.
hits = collection.query(
query_texts=["What are the early termination conditions?"],
n_results=5,
where={"file_id": result.chunksets[0].file_id},
where_document={"$contains": "termination"},
)POMA validates the payload up front (a corrupted or mislabeled upload 422s immediately), reads md over text so no structure is lost, neutralizes the dead  image references and counts them in content_metadata, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "llamaparse" or "none". After retrieval, merge hits instead of concatenating: chunksets from the same section share ancestor breadcrumbs, and deduplicating that lineage assembles one prompt-ready cheatsheet — 337 tokens of context versus 1,542 for a recursive-splitter baseline on our reference legal-document benchmark, zero information loss (methodology).
Metadata mapping: POMA fields → Chroma primitives
| POMA chunk field | Chroma primitive | What it enables |
|---|---|---|
to_embed | documents entry, embedded by the collection's embedding function | semantic query_texts retrieval |
file_id | metadatas key → where={"file_id": ...} | scope queries to one document |
page (on the archive's chunk records, derived from LlamaParse pages[].page; take the chunkset's leaf chunk) | metadatas key → where={"page": {"$gte": ...}} | page-cited answers, page-range filters |
depth (of the chunkset's leaf chunk) | metadatas key | filter/re-rank by hierarchy level |
chunkset_index | metadatas key + part of the entry id | stable IDs, ordering at assembly time |
| chunkset lineage | breadcrumbs inside the document text + the .poma archive | cheatsheet assembly; portability to any production store |
The last row is the strategic one. Because the .poma archive — not the Chroma directory — is the source of truth, the collection above is a projection. Moving to Qdrant, pgvector, or Milvus later is a re-upsert of the same chunksets with the same metadata: no re-parsing, no re-chunking, no re-running your evaluation suite from scratch.
Frequently asked questions
How do I get LlamaParse results into Chroma for RAG?
Save the raw JSON result, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), then collection.add the chunksets: to_embed strings as documents, file_id/page/depth/chunkset_index as metadata — page comes from the archive's chunk records (the chunkset's leaf chunk), depth from the matching typed chunk. Query with query_texts plus where and where_document.
Should each LlamaParse page become one Chroma document?
No — pages[].md makes it tempting, but page-sized vectors blur every section on the page, n_results=5 returns five walls of text, and sections spanning page breaks are severed. Chunk by heading hierarchy instead; chunksets stay self-explanatory regardless of page boundaries.
What where filters work for LlamaParse content in Chroma?
where={"file_id": ...} for document scope, $gte/$in for page ranges and multi-document sets, where_document={"$contains": ...} for content matching — provided the metadata was written at add time. Concatenate-then-split pipelines lose the page numbers and have nothing to filter on.
Do LlamaParse image references end up in the Chroma collection?
Naively, yes — the saved JSON's dead  links get embedded and can even match $contains filters. POMA neutralizes each reference and counts it in content_metadata, so figure loss is visible and quantified, never noise in the collection.
Can I move a Chroma prototype built on LlamaParse output to production without re-chunking?
Yes — the .poma archive holds the chunks, chunksets, and metadata as the portable source of truth. Chroma is one projection; Qdrant, Pinecone, Weaviate, pgvector, and Milvus are others. Migration is a re-upsert, and your validated retrieval behavior carries over.
Related recipes
Same parser, different store: LlamaParse → Qdrant · LlamaParse → pgvector · LlamaParse → Milvus
Same store, different parser: Mistral OCR → Chroma · Docling → Chroma · Marker → Chroma
Foundations: The Optimal Chunker for LlamaParse · The Optimal Chunks for Chroma · All pipeline recipes · RAG chunking guide