Source: http://www.poma-ai.com/docs/pipelines/llamaparse-to-mongodb-atlas

# The Missing Link Between LlamaParse and Optimal Retrieval in MongoDB Atlas

<ByAuthor />

**The short answer:** LlamaParse gives you excellent per-page markdown; MongoDB Atlas gives you vector search folded directly into the aggregation pipeline you already run. Wired together naively — split each page, embed, `insert_one` — they still produce mediocre RAG, because nothing in between rebuilds cross-page hierarchy or guarantees the metadata `$vectorSearch` needs actually comes back. The missing link is POMA: `PrimeCut().ingest()` consumes the raw LlamaParse JSON (auto-detected), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets) as a portable `.poma` archive, and `vektoria`'s `records_from_archive()` + `pymongo` write content-free documents — `embedding`, `metadata.file_id`, `metadata.chunkset_index` — while chunkset content lives on a volume. Retrieval runs Atlas's own `$vectorSearch` stage, then `assemble()` reassembles prompt-ready context.

## What each end of the pipeline actually provides

**LlamaParse** returns `pages[]`, each with a `page` number, an `md` string (markdown, tables inline), a `text` flattening, and an `images` list whose bytes stay server-side — a saved result JSON carries only `![](name)` references. What it doesn't provide: cross-page hierarchy (a `##` heading on page 41 has no link to the `#` chapter that opened on page 3) or retrieval units. Details: [The Optimal Chunker for LlamaParse](/optimal-chunker-llamaparse).

**MongoDB Atlas** provides the `$vectorSearch` aggregation stage (Atlas-specific, not self-hosted MongoDB), `numCandidates`/`limit` tuning, and compound hybrid search with Atlas Search's BM25-style text matching. What it doesn't provide: any default that surfaces fields you didn't `$project` — the aggregation pipeline drops what it isn't told to keep. Details: [MongoDB Atlas Chunking Strategy for RAG](/optimal-chunks-mongodb-atlas).

## The naive wiring, and where it breaks

The common recipe — split each `pages[].md` entry on its own, embed, `insert_one({"embedding": vec, "text": chunk})` — breaks in a pair-specific way that compounds two quirks at once:

- **Splitting per page bakes in LlamaParse's independence.** LlamaParse never links headings across pages, so treating each page's `md` as a standalone unit — instead of rebuilding the hierarchy first — locks in orphan fragments as your permanent document shape. There is no later step that recovers the chapter a page-41 clause belongs to.
- **Flat fields instead of a projected `metadata` sub-document.** A naive insert writes `file_id`/`chunkset_index` as top-level fields, or skips them, rather than the `metadata` sub-document `$project` is built to surface. Either way, once you add the `$vectorSearch` stage to your pipeline, an aggregation without an explicit `$project` of `metadata` returns only what MongoDB decides to keep — usually just the default `_id` and requested score meta.
- **The two failures mask each other.** Fixing the `$project` gotcha alone gets you `assemble()` results — but every "hit" is still an orphaned per-page fragment with no lineage to deduplicate, because the hierarchy rebuild PrimeCut does was skipped in the first place.

## The pipeline, end to end

```bash
pip install llama-parse pymongo
```

```python
import json
import os
from llama_parse import LlamaParse
from pymongo import MongoClient
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder

# 1. Your existing LlamaParse call — unchanged. Save the raw result JSON.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
    json.dump(json_result[0], f)

# 2. The missing link — raw LlamaParse JSON in, a portable .poma archive out.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.llamaparse.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
mongo = MongoClient(os.environ["MONGODB_ATLAS_URI"])
col = mongo["poma"]["chunksets"]

# Atlas vector index (numDimensions, similarity) is created once via the Atlas UI/API,
# named "poma_vec" here — not through the pymongo driver itself.

# 3. Content-free ingest — the document holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
for r in records:
    vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
                                          "chunks": r.payload["chunks"], "text": r.text})
    col.insert_one({
        "_id": r.id,
        "embedding": embedder.embed([r.text])[0],
        "metadata": {"file_id": r.payload["file_id"], "chunkset_index": r.payload["chunkset_index"]},
    })

# 4. $vectorSearch retrieval — $project the metadata explicitly, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
hits = list(col.aggregate([
    {"$vectorSearch": {"index": "poma_vec", "path": "embedding",
                       "queryVector": qv, "numCandidates": 200, "limit": 10}},
    {"$project": {"_id": 1, "metadata": 1, "score": {"$meta": "vectorSearchScore"}}},
]))
context = assemble(hits, volume=vol)  # -> [{"file_id", "content"}, ...]
```

PrimeCut validates the payload shape up front (a mislabeled upload 422s immediately), prefers `md` over `text`, neutralizes LlamaParse's dead image references into visible, counted markers, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set `external_ocr_source` to `"llamaparse"` or `"none"`.

## Metadata mapping: POMA fields → MongoDB Atlas primitives

| POMA chunk field | MongoDB Atlas primitive | What it enables |
| --- | --- | --- |
| `to_embed` | `embedding` field, covered by the `$vectorSearch` index | dense retrieval via `$vectorSearch` |
| `file_id` | `metadata.file_id`, ordinary field | `$match`/`filter` scoping alongside the vector stage |
| `chunkset_index` | `metadata.chunkset_index`, ordinary field | deterministic id + lineage for `assemble()`, surfaced only via `$project` |
| chunkset content (`text`, `chunks`) | volume, not a document field | keeps documents small; content fetched at assemble time |

## Frequently asked questions

### How do I get LlamaParse results into MongoDB Atlas for RAG?

Save the raw result JSON, run `PrimeCut().ingest()` on it (auto-detected, archived to `.poma`), then insert `records_from_archive()` output as documents with an `embedding` field and a `metadata` sub-document. Retrieve with `$vectorSearch` + `$project`, then `assemble(hits, volume=vol)`.

### Why not just split LlamaParse's md per page and insert into MongoDB Atlas directly?

Each LlamaParse page's `md` is independent, so splitting per page without rebuilding the cross-page hierarchy first bakes orphan fragments into your document shape permanently — Atlas ranks them with the same confidence as real chunksets.

### What document shape does MongoDB Atlas need for LlamaParse chunksets?

One document per chunkset: an `embedding` field covered by an Atlas Vector Search index (`numDimensions`, `similarity: "cosine"`), plus a `metadata` sub-document with `file_id`/`chunkset_index` for `$match` scoping.

### Why does assemble() return nothing from my MongoDB Atlas query after ingesting LlamaParse content?

`$vectorSearch` must be followed by a `$project` that explicitly includes `metadata`. This is worse with a naive LlamaParse pipeline, which often never wraps `file_id`/`chunkset_index` into a `metadata` sub-document to begin with — there's nothing to project even after you fix the aggregation.

### Does MongoDB Atlas Vector Search work with LlamaParse's offloaded images?

Yes — PrimeCut neutralizes LlamaParse's dead `![](name)` references into visible, counted markers before chunking, so no document inserted into Atlas represents an unresolved image link as real content.

## Related recipes

Same parser, different store: [LlamaParse → Redis](/pipelines/llamaparse-to-redis) · [LlamaParse → LanceDB](/pipelines/llamaparse-to-lancedb) · [LlamaParse → Pinecone](/pipelines/llamaparse-to-pinecone)

Foundations: [The Optimal Chunker for LlamaParse](/optimal-chunker-llamaparse) · [MongoDB Atlas Chunking Strategy for RAG](/optimal-chunks-mongodb-atlas) · [All pipeline recipes](/pipelines/) · [RAG architecture guide](/guides/rag-architecture/)