LlamaParse to MongoDB Atlas: Chunking for RAG
The short answer: LlamaParse gives you excellent per-page markdown; MongoDB Atlas gives you vector search folded directly into the aggregation pipeline you already run. Wired together naively — split each page, embed, insert_one — they still produce mediocre RAG, because nothing in between rebuilds cross-page hierarchy or guarantees the metadata $vectorSearch needs actually comes back. The missing link is POMA: PrimeCut().ingest() consumes the raw LlamaParse JSON (auto-detected), emits hierarchy-preserving chunksets as a portable .poma archive, and vektoria's records_from_archive() + pymongo write content-free documents — embedding, metadata.file_id, metadata.chunkset_index — while chunkset content lives on a volume. Retrieval runs Atlas's own $vectorSearch stage, then assemble() reassembles prompt-ready context.
What each end of the pipeline actually provides
LlamaParse returns pages[], each with a page number, an md string (markdown, tables inline), a text flattening, and an images list whose bytes stay server-side — a saved result JSON carries only  references. What it doesn't provide: cross-page hierarchy (a ## heading on page 41 has no link to the # chapter that opened on page 3) or retrieval units. Details: The Optimal Chunker for LlamaParse.
MongoDB Atlas provides the $vectorSearch aggregation stage (Atlas-specific, not self-hosted MongoDB), numCandidates/limit tuning, and compound hybrid search with Atlas Search's BM25-style text matching. What it doesn't provide: any default that surfaces fields you didn't $project — the aggregation pipeline drops what it isn't told to keep. Details: MongoDB Atlas Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — split each pages[].md entry on its own, embed, insert_one({"embedding": vec, "text": chunk}) — breaks in a pair-specific way that compounds two quirks at once:
- Splitting per page bakes in LlamaParse's independence. LlamaParse never links headings across pages, so treating each page's
mdas a standalone unit — instead of rebuilding the hierarchy first — locks in orphan fragments as your permanent document shape. There is no later step that recovers the chapter a page-41 clause belongs to. - Flat fields instead of a projected
metadatasub-document. A naive insert writesfile_id/chunkset_indexas top-level fields, or skips them, rather than themetadatasub-document$projectis built to surface. Either way, once you add the$vectorSearchstage to your pipeline, an aggregation without an explicit$projectofmetadatareturns only what MongoDB decides to keep — usually just the default_idand requested score meta. - The two failures mask each other. Fixing the
$projectgotcha alone gets youassemble()results — but every "hit" is still an orphaned per-page fragment with no lineage to deduplicate, because the hierarchy rebuild PrimeCut does was skipped in the first place.
The pipeline, end to end
pip install llama-parse pymongoimport json
import os
from llama_parse import LlamaParse
from pymongo import MongoClient
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
# 1. Your existing LlamaParse call — unchanged. Save the raw result JSON.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
json.dump(json_result[0], f)
# 2. The missing link — raw LlamaParse JSON in, a portable .poma archive out.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.llamaparse.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
mongo = MongoClient(os.environ["MONGODB_ATLAS_URI"])
col = mongo["poma"]["chunksets"]
# Atlas vector index (numDimensions, similarity) is created once via the Atlas UI/API,
# named "poma_vec" here — not through the pymongo driver itself.
# 3. Content-free ingest — the document holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
for r in records:
vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
"chunks": r.payload["chunks"], "text": r.text})
col.insert_one({
"_id": r.id,
"embedding": embedder.embed([r.text])[0],
"metadata": {"file_id": r.payload["file_id"], "chunkset_index": r.payload["chunkset_index"]},
})
# 4. $vectorSearch retrieval — $project the metadata explicitly, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
hits = list(col.aggregate([
{"$vectorSearch": {"index": "poma_vec", "path": "embedding",
"queryVector": qv, "numCandidates": 200, "limit": 10}},
{"$project": {"_id": 1, "metadata": 1, "score": {"$meta": "vectorSearchScore"}}},
]))
context = assemble(hits, volume=vol) # -> [{"file_id", "content"}, ...]PrimeCut validates the payload shape up front (a mislabeled upload 422s immediately), prefers md over text, neutralizes LlamaParse's dead image references into visible, counted markers, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "llamaparse" or "none".
Metadata mapping: POMA fields → MongoDB Atlas primitives
| POMA chunk field | MongoDB Atlas primitive | What it enables |
|---|---|---|
to_embed | embedding field, covered by the $vectorSearch index | dense retrieval via $vectorSearch |
file_id | metadata.file_id, ordinary field | $match/filter scoping alongside the vector stage |
chunkset_index | metadata.chunkset_index, ordinary field | deterministic id + lineage for assemble(), surfaced only via $project |
chunkset content (text, chunks) | volume, not a document field | keeps documents small; content fetched at assemble time |
Frequently asked questions
How do I get LlamaParse results into MongoDB Atlas for RAG?
Save the raw result JSON, run PrimeCut().ingest() on it (auto-detected, archived to .poma), then insert records_from_archive() output as documents with an embedding field and a metadata sub-document. Retrieve with $vectorSearch + $project, then assemble(hits, volume=vol).
Why not just split LlamaParse's md per page and insert into MongoDB Atlas directly?
Each LlamaParse page's md is independent, so splitting per page without rebuilding the cross-page hierarchy first bakes orphan fragments into your document shape permanently — Atlas ranks them with the same confidence as real chunksets.
What document shape does MongoDB Atlas need for LlamaParse chunksets?
One document per chunkset: an embedding field covered by an Atlas Vector Search index (numDimensions, similarity: "cosine"), plus a metadata sub-document with file_id/chunkset_index for $match scoping.
Why does assemble() return nothing from my MongoDB Atlas query after ingesting LlamaParse content?
$vectorSearch must be followed by a $project that explicitly includes metadata. This is worse with a naive LlamaParse pipeline, which often never wraps file_id/chunkset_index into a metadata sub-document to begin with — there's nothing to project even after you fix the aggregation.
Does MongoDB Atlas Vector Search work with LlamaParse's offloaded images?
Yes — PrimeCut neutralizes LlamaParse's dead  references into visible, counted markers before chunking, so no document inserted into Atlas represents an unresolved image link as real content.
Related recipes
Same parser, different store: LlamaParse → Redis · LlamaParse → LanceDB · LlamaParse → Pinecone
Foundations: The Optimal Chunker for LlamaParse · MongoDB Atlas Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide