Source: http://www.poma-ai.com/docs/pipelines/docling-to-mongodb-atlas

# The Missing Link Between Docling and Optimal Retrieval in MongoDB Atlas

<ByAuthor />

**The short answer:** Docling gives you an explicit, typed document tree; MongoDB Atlas gives you vector search folded into the aggregation pipeline you already run. Wired together the common way — flatten to markdown, split, embed, insert one document per fragment — both tools end up worse than either alone, because nothing rebuilds cross-page hierarchy or keeps the metadata field flowing through the pipeline. The missing link is POMA: `PrimeCut().ingest()` consumes the DoclingDocument tree directly, emits hierarchy-preserving [chunksets](/learn/chunking/chunksets), and `vektoria` writes them into Atlas as content-free documents — vector plus `{file_id, chunkset_index}` only — while the actual text lives on a volume. Retrieval runs `$vectorSearch` with an explicit `$project`, then `assemble()` reconstructs prompt-ready cheatsheets.

## What each end of the pipeline actually provides

**Docling** (IBM's open-source converter, also served via docling-serve) returns a typed **DoclingDocument**: `texts`/`tables`/`pictures`/`groups`, with `section_header` items carrying an explicit numeric `level` and repeating page furniture pre-isolated into a `furniture` group labeled `page_header`/`page_footer`. What it doesn't provide is a retrieval unit — its own `HybridChunker` packs tree items into token windows sized for an embedding model, which controls length, not whether a retrieved passage carries the heading path that explains it. Details: [The Optimal Chunker for Docling](/optimal-chunker-docling).

**MongoDB Atlas** provides `$vectorSearch` as an ordinary aggregation stage — a search-index type declared once (`path`, `numDimensions`, `similarity`), then queried inside the same pipeline you use for `$match` and `$project`. What it doesn't provide is any opinion about what a document should represent, and its aggregation model has a sharp edge: fields you don't explicitly project don't come back. Details: [MongoDB Atlas Chunking Strategy for RAG](/optimal-chunks-mongodb-atlas).

## The naive wiring, and where it breaks

The common recipe — `doc.export_to_markdown()` → `RecursiveCharacterTextSplitter` → `insert_one` per fragment — breaks in a pair-specific way:

- **Docling's `section_header` levels are discarded at export.** The tree already told you a passage lives under *Termination clauses* under *Master Services Agreement*; flattening to markdown reduces that to `##` glyphs the splitter ignores, cutting wherever the character count lands.
- **Furniture Docling already isolated re-enters the text stream.** The `furniture` group and `page_header`/`page_footer` labels exist so downstream tools skip re-detecting running noise — flattening throws that away, and page numbers end up embedded and indexed.
- **Overlap inflates the vector index for no retrieval benefit.** Splitter overlap duplicates every boundary span, and Atlas's vector index cost scales with document count like any other collection index — redundant near-duplicate vectors crowd top-k with no upside.
- **The aggregation pipeline silently drops the field retrieval depends on.** A splitter-based pipeline typically writes each fragment as `{text, embedding}` with no thought given to a `metadata` field, and even when one exists, a `$vectorSearch` stage with no trailing `$project` on it means MongoDB never returns `file_id`/`chunkset_index` — the query succeeds, `assemble()` gets nothing, and this failure mode looks identical whether the vectors came from Docling, Textract, or anything else, which makes it easy to misdiagnose as a Docling parsing problem when it isn't.

## The pipeline, end to end

```bash
pip install docling pymongo
```

```python
import json
import os
from docling.document_converter import DocumentConverter
from pymongo import MongoClient
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder

# 1. Your existing Docling conversion — unchanged. Save the tree as JSON.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

# 2. The missing link — raw DoclingDocument JSON in, a portable .poma archive out.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.docling.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
mongo = MongoClient(os.environ["MONGODB_ATLAS_URI"])
col = mongo["poma"]["chunksets"]

# Atlas vector index (numDimensions, similarity) is created once via the Atlas UI/API,
# named "poma_vec" here — not through the pymongo driver itself.

# 3. Content-free ingest — Atlas holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
for r in records:
    vol.write_doc(r.payload["file_id"], {"file_id": r.payload["file_id"],
                                          "chunks": r.payload["chunks"], "text": r.text})
    col.insert_one({
        "_id": r.id,
        "embedding": embedder.embed([r.text])[0],
        "metadata": {"file_id": r.payload["file_id"], "chunkset_index": r.payload["chunkset_index"]},
    })

# 4. $vectorSearch retrieval — $project the metadata explicitly, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
hits = list(col.aggregate([
    {"$vectorSearch": {"index": "poma_vec", "path": "embedding",
                       "queryVector": qv, "numCandidates": 200, "limit": 10}},
    {"$project": {"_id": 1, "metadata": 1, "score": {"$meta": "vectorSearchScore"}}},
]))
context = assemble(hits, volume=vol)  # -> [{"file_id", "content"}, ...]
```

`records_from_archive` walks the `.poma` archive produced by ingest — the same archive that's the volume's source of truth — and returns one `Record` per chunkset: a deterministic `id`, the `to_embed` text, and a `payload` carrying `file_id`, `chunkset_index`, and the member chunk list. No `push()` connector exists for Atlas yet; ingest here is the native `pymongo` client, shaped exactly the way vektoria's `assemble()` already expects on the way back out.

## Metadata mapping: POMA fields → MongoDB Atlas primitives

| POMA record field | Atlas primitive | What it enables |
| --- | --- | --- |
| `id` (`chunkset_uuid`) | document `_id` | deterministic key — safe re-ingest, no duplicate documents |
| `text` (`to_embed`) | `embedding` field, covered by the Atlas Vector Search index | the `$vectorSearch` similarity query itself |
| `payload["file_id"]` | `metadata.file_id` | document-scoping via `$match`, once projected |
| `payload["chunkset_index"]` | `metadata.chunkset_index` | routes a hit back to its content on the volume |
| chunkset content | volume (`Volume.write_doc`), never Atlas | smaller documents; citations reconstructed at retrieval |

## Frequently asked questions

### How do I get Docling output into MongoDB Atlas for RAG?

Export the DoclingDocument tree with `export_to_dict` and save it as JSON. `PrimeCut().ingest()` auto-detects the shape and rebuilds hierarchy into chunksets; write each one's content to a `vektoria` `Volume`, embed its `to_embed` text, and insert a document with the vector plus a `metadata` field carrying `file_id`/`chunkset_index`. Query with `$vectorSearch`, project `metadata` explicitly, then `assemble()`.

### Why not just export Docling to markdown and embed it into MongoDB Atlas directly?

`export_to_markdown()` collapses explicit `section_header` levels and pre-isolated furniture into display text a splitter must re-guess, and — specific to Atlas — a `$vectorSearch` stage with no trailing `$project` silently drops the metadata field a splitter-based pipeline never thought to add.

### What MongoDB Atlas fields should Docling chunksets carry?

An `embedding` field covered by the vector index, plus a `metadata` field holding `file_id`/`chunkset_index`. Docling's hierarchy is resolved upstream by PrimeCut, so it lives in the chunkset text on the volume, not a separate document field.

### Why does POMA assemble return nothing from my MongoDB Atlas results after ingesting Docling chunksets?

`$vectorSearch` must be followed by a `$project` that explicitly includes `metadata` — the aggregation pipeline drops unprojected fields regardless of which parser produced the vectors, so `assemble()` receives hits with no `file_id`/`chunkset_index` to resolve.

### Does MongoDB Atlas support hybrid search for Docling-parsed documents?

Yes — Atlas Search's compound query type combines BM25-style text matching with `$vectorSearch` results in the same pipeline, useful for Docling-sourced documents where defined terms and clause numbers are exact tokens dense embeddings alone tend to blur.

## Related recipes

Same parser, different store: [Docling → Qdrant](/pipelines/docling-to-qdrant) · [Docling → Pinecone](/pipelines/docling-to-pinecone) · [Docling → Weaviate](/pipelines/docling-to-weaviate)

Foundations: [The Optimal Chunker for Docling](/optimal-chunker-docling) · [MongoDB Atlas Chunking Strategy for RAG](/optimal-chunks-mongodb-atlas) · [All pipeline recipes](/pipelines/) · [RAG architecture guide](/guides/rag-architecture/)