Marker to MongoDB Atlas: Chunking for RAG
The short answer: Marker gives you fast, local PDF parsing with real recovered structure; MongoDB Atlas gives you vector search folded directly into the aggregation pipeline you already run. Wired together naively — flatten the markdown, split, embed, insert, then $vectorSearch with no follow-up stage — the pipeline drops every figure before Atlas sees a document, and the aggregation stage silently drops metadata it wasn't told to keep. The missing link is POMA: PrimeCut().ingest() consumes the saved Marker JSON (the Document block tree auto-detects), splices the side-channel image bytes back in, and rebuilds the cross-page hierarchy into chunksets. vektoria keeps Atlas content-free — embeddings and routing metadata only — and assemble() turns a correctly-projected $vectorSearch result into prompt-ready context.
What each end of the pipeline actually provides
Marker (by Datalab / VikParuchuri) runs on your own hardware and turns PDFs into clean structured output. With the JSON renderer (--output_format json) you get a Document block tree — Page children, per-block HTML, full <table> elements with row and column spans. What it doesn't provide: cross-page hierarchy, retrieval units, or inline images — figure bytes live in a side-channel images dict keyed by name, with only  references left in the text. Details: The Optimal Chunker for Marker.
MongoDB Atlas provides a $vectorSearch aggregation stage on an Atlas-managed vector index, combinable with $match or Atlas Search's compound query type for hybrid, over documents you already model in your own collections. What it doesn't provide: any opinion about what a document should hold, or a $project by default — the aggregation pipeline drops any field you didn't explicitly ask for, metadata included. Details: MongoDB Atlas Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — run Marker's default markdown output, RecursiveCharacterTextSplitter, embed, insert_one with only an embedding field, then $vectorSearch with no follow-up stage — breaks in a way specific to this pair:
- Every figure vanishes before Atlas. Marker's images live in the side-channel
imagesdict; the markdown carries onlyreferences. A markdown-only pipeline embeds those dangling references as literal text, so the chart that answers a question was never inserted as a document at all. $vectorSearchwith no$projectreturns bare IDs. MongoDB's aggregation pipeline only returns fields you explicitly project; a query that ends at$vectorSearchand skips a metadata$projecthands back scores with no way to know which document or chunkset a hit belongs to —assemble()gets a non-empty result with nothing extractable in it.- Page numbers never make it into a filterable field. Marker's default markdown output has no page boundaries, so there's no ordinary document field to
$matchon for a page-scoped query — the JSON renderer's Page blocks are the only source for one. - Overlap inflates the collection. Splitter overlap embeds every boundary span twice, and Atlas's vector index cost scales with document count like any other collection index.
The pipeline, end to end
# 1. Marker — your existing run, unchanged. The JSON renderer keeps the
# Document block tree that POMA auto-detects, pages included.
marker_single contract.pdf --output_format json --output_dir out/import os
from pymongo import MongoClient
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
# 2. The missing link — raw Marker JSON in, .poma archive out (chunks +
# chunksets, image bytes spliced back into their references).
client = PrimeCut() # reads POMA_API_KEY
client.ingest("out/contract/contract.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
mongo = MongoClient(os.environ["MONGODB_ATLAS_URI"])
col = mongo["poma"]["chunksets"]
# Atlas vector index (numDimensions, similarity) is created once via the
# Atlas UI/API, named "poma_vec" here — not through the pymongo driver itself.
# 3. Content-free ingest — Atlas holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
for r in records:
vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
"chunks": r.payload["chunks"], "text": r.text})
col.insert_one({
"_id": r.id,
"embedding": embedder.embed([r.text])[0],
"metadata": {"file_id": r.payload["file_id"], "chunkset_index": r.payload["chunkset_index"]},
})
# 4. $vectorSearch retrieval — $project the metadata explicitly, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
hits = list(col.aggregate([
{"$vectorSearch": {"index": "poma_vec", "path": "embedding",
"queryVector": qv, "numCandidates": 200, "limit": 10}},
{"$project": {"_id": 1, "metadata": 1, "score": {"$meta": "vectorSearchScore"}}},
]))
context = assemble(hits, volume=vol) # -> [{"file_id", "content"}, ...]POMA validates the payload shape up front — the JSON Document block tree is the strict auto-route fingerprint, and a corrupted or mislabeled upload 422s immediately instead of degrading the index. If what you saved is one of Marker's bare markdown shapes, declare it explicitly with external_ocr_source="marker". On our reference legal document, this chunking answers the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter — methodology in Document Ingestion & Chunking for RAG.
Metadata mapping: POMA fields → MongoDB Atlas primitives
| POMA chunk field | MongoDB Atlas primitive | What it enables |
|---|---|---|
to_embed | embedding field, covered by the $vectorSearch index | similarity search over the embedded chunkset |
file_id | metadata.file_id, ordinary document field | $match/filter scoping to one document |
chunkset_index | metadata.chunkset_index | assemble() lookup key, stable ordering |
page (JSON renderer only) | volume document, not a field | content-free retrieval, cheatsheet assembly |
| chunkset content | not in the document — lives on a Volume | small documents, reassembled at retrieval |
Frequently asked questions
How do I get Marker output into MongoDB Atlas for RAG?
Run Marker with the JSON renderer and save the result; hand it to PrimeCut().ingest(), which auto-detects the Document block tree, splices images back in, and writes a .poma archive of chunks and chunksets. Pull records with records_from_archive(), embed each record's to_embed text, and insert one document per record with an embedding field plus a metadata field holding file_id and chunkset_index. Query with $vectorSearch, $project the metadata explicitly, and hand the aggregation result to assemble().
Do Marker's extracted images make it into MongoDB Atlas?
Only if something reunites them first. Marker parks figure bytes in a side-channel images dict and leaves bare  references in the markdown, so a pipeline that embeds only the markdown inserts documents with zero figures represented in the vector field. POMA splices the bytes back into their references as data URIs and describes each figure into searchable text before embedder.embed() ever runs, so the figure becomes a real embedded document. References without bytes are neutralized and counted, never silently dropped.
Why does POMA assemble return nothing from my MongoDB Atlas results after ingesting Marker output?
Almost always a missing $project. The $vectorSearch aggregation stage must be followed by a $project that explicitly includes the metadata field holding file_id and chunkset_index — MongoDB's aggregation pipeline drops any field you didn't ask for, unlike a plain find(). If the ingest step wrote metadata correctly but the query stage doesn't project it, assemble() gets a non-empty result with nothing to extract, and comes back empty.
What MongoDB Atlas fields should Marker chunks carry?
An embedding field covered by the Atlas vector index, and a metadata field holding file_id and chunkset_index — the same fields records_from_archive() hands you after ingesting Marker's JSON block tree. Page numbers as an ordinary indexed field only exist if you ran Marker's JSON renderer; the default markdown output has no page boundaries to store at all.
Does Marker's JSON renderer matter for a MongoDB Atlas pipeline?
Yes. The JSON renderer's Document block tree is what POMA auto-detects and what preserves per-page structure, which is the only way a page field ends up in your Atlas documents at all; Marker's default markdown output loses page boundaries entirely. The JSON renderer also keeps full HTML tables intact instead of collapsing them into one flat string, which matters once that content is deduplicated into a cheatsheet at retrieval time.
Related recipes
Same parser, different store: Marker → Weaviate · Marker → pgvector · Marker → Milvus
Foundations: The Optimal Chunker for Marker · MongoDB Atlas Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide