Source: http://www.poma-ai.com/docs/pipelines/docling-to-chroma

# The Missing Link Between Docling and Optimal Retrieval in Chroma

<ByAuthor />

**The short answer:** Docling and Chroma are the natural local, open-source pairing — a typed document tree from a parser you run yourself, into a vector store that lives on your disk. Wired together naively — flatten, split, `collection.add` — the demo works and the mistake ships, because the chunking decision made at prototype time is the one that sticks. The missing link is POMA: `PrimeCut().ingest()` consumes the saved DoclingDocument JSON (auto-detected), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets) with `where`-filterable metadata, and leaves a portable `.poma` archive so the prototype's retrieval behavior moves to any production store unchanged.

## What each end of the pipeline actually provides

**Docling** (IBM's open-source converter, also served via docling-serve) returns the **DoclingDocument**: a typed `texts`/`tables`/`pictures`/`groups` tree in which `section_header` items carry an explicit numeric `level`, tables are cell grids, and repeating page furniture is pre-isolated in the `furniture` group with `page_header`/`page_footer` labels. What it doesn't provide: retrieval units — its HybridChunker packs the tree into token windows, which controls length, not meaning. Details: [The Optimal Chunker for Docling](/optimal-chunker-docling).

**Chroma** provides the fastest path from zero to a working retrieval loop: `chromadb.PersistentClient(path=...)` gives you a durable local collection with no server, the same API scales to client/server mode, `where` filters act on scalar metadata, `where_document` adds content conditions, and default embedding functions mean `documents=` is all you need to pass. What it doesn't provide: any opinion about what a document should contain. Details: [The Optimal Chunks for the Best Retrieval in Chroma](/optimal-chunks-chroma).

## The naive wiring, and where it breaks

The common recipe is an afternoon notebook — `export_to_markdown()` → `RecursiveCharacterTextSplitter` → `collection.add()` — and it breaks in a pair-specific way: **this is the series' most local, most open-source pipeline, which makes it the one most likely to be built as a quick prototype whose chunking then ships.**

- **The demo hides the damage.** At notebook scale, Chroma makes fragments look retrievable — the toy queries hit. The splitter becomes the least-examined, most load-bearing line in the stack.
- **Docling's structure is gone before Chroma sees it.** Explicit `section_header` levels, typed tables, and pre-isolated furniture flatten into display text — so there is no `file_id`/`page`/`depth` metadata, and `where` filters (Chroma's main retrieval lever beyond similarity) have nothing to act on.
- **Overlap fills the collection with near-duplicates** that crowd `n_results` — five results collapse into two distinct passages.
- **Graduation day is a rewrite.** When production lands on a server-grade store, fragments chunked for the notebook must be re-chunked, re-embedded, and re-evaluated — the prototype validated nothing that survives.

## The pipeline, end to end

```bash
pip install docling poma chromadb
```

```python
import json

import chromadb
from docling.document_converter import DocumentConverter
from poma import PrimeCut

# 1. Docling — local parsing, your existing conversion unchanged.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

# 2. The missing link — DoclingDocument tree in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("contract.docling.json")  # Docling shape auto-detected

# 3. Chroma — a durable local collection; same API in client/server mode.
chroma = chromadb.PersistentClient(path="./chroma")
collection = chroma.get_or_create_collection("contracts")

# 4. Add chunksets: Chroma embeds `documents` with its default embedding
#    function (or bring your own); hierarchy lands as filterable metadata.
collection.add(
    documents=[cs.to_embed for cs in result.chunksets],
    metadatas=[
        {"file_id": cs.file_id, "page": cs.page, "depth": cs.depth}
        for cs in result.chunksets
    ],
    ids=[f"{cs.file_id}-{i}" for i, cs in enumerate(result.chunksets)],
)

# 5. Query, scoped by the hierarchy metadata.
hits = collection.query(
    query_texts=["What are the early termination conditions?"],
    n_results=5,
    where={"file_id": result.chunksets[0].file_id},
)
```

POMA validates the payload up front (a corrupted or mislabeled upload 422s immediately, never a silently degraded collection), honors the furniture Docling already isolated, keeps tables as HTML, and accepts docling-serve `md_content` envelopes via a markdown passthrough. To force or suppress detection, set `external_ocr_source` to `"docling"` or `"none"`.

The ingest also produces a portable `.poma` archive — chunks, chunksets, and hierarchy metadata as the source of truth on your disk, with Chroma as one projection of it. Retrieved chunksets merge into a deduplicated, prompt-ready cheatsheet — on the reference legal-document benchmark, **337 tokens** of context versus **1,542** for a recursive-splitter baseline, with zero information loss ([methodology](/document-ingestion-chunking-rag)).

## Metadata mapping: POMA fields → Chroma primitives

| POMA chunk field | Chroma primitive | What it enables |
| --- | --- | --- |
| `to_embed` | `documents` (embedded by the default function, or your own) | semantic retrieval over self-explanatory units |
| `file_id` | metadata scalar → `where={"file_id": ...}` | per-document scoping |
| `page` | metadata scalar (int) → `$gte`/`$lte` filters | page-cited answers, page-range queries |
| `depth` (from Docling `section_header` `level`) | metadata scalar (int) | exclude deep appendix material via `where` |
| `chunk_index` | metadata scalar / stable `ids` suffix | deterministic ordering at assembly time |
| chunkset lineage | `.poma` archive (portable source of truth) | cheatsheet assembly + store-to-store portability |

## Frequently asked questions

### How do I load Docling output into ChromaDB?

Save `export_to_dict()` as JSON and run `PrimeCut().ingest()` on it — the `schema_name` fingerprint routes it through the BYOCR seam. Then `collection.add` the chunksets: `to_embed` as `documents` (Chroma embeds them) and `file_id`/`page`/`depth` as metadata. Query with `query_texts` plus a `where` filter.

### Can I run Docling and Chroma together in a local RAG stack?

Mostly — Docling parses locally and `PersistentClient` stores and searches locally with no server. The chunking step runs through POMA's API and returns chunks, chunksets, and a portable `.poma` archive that lives on your disk. Everything you store, query, and evaluate stays local.

### What Chroma metadata should Docling chunks carry?

`file_id`, `page`, `depth`, `chunk_index` — flat scalars, exactly what Chroma metadata requires. `depth` comes from Docling's explicit `section_header` levels, so depth-based `where` filters act on detected hierarchy; `where_document` adds content conditions. None of it exists if the tree was flattened first.

### Will a Docling plus Chroma prototype survive the move to a production vector database?

Yes, if the chunks live outside Chroma as a portable artifact. The `.poma` archive is the source of truth; moving to Qdrant, pgvector, Weaviate, Milvus, or Pinecone is a re-upsert of the same chunksets with the same metadata, so validated retrieval behavior carries over.

### Why not just export Docling markdown, split it, and add it to Chroma?

Because the prototype's chunking ships. Flattening discards the heading levels, tables, and furniture labels Docling recovered; the demo still works at small scale, but `where` filters have no metadata, overlap crowds `n_results` with near-duplicates, and re-chunking after launch means re-embedding and re-evaluating everything.

## Related recipes

Same parser, different store: [Docling → Qdrant](/pipelines/docling-to-qdrant) · [Docling → pgvector](/pipelines/docling-to-pgvector) · [Docling → Milvus](/pipelines/docling-to-milvus)

Same store, different parser: [Marker → Chroma](/pipelines/marker-to-chroma) · [Unstructured.io → Chroma](/pipelines/unstructured-to-chroma) · [Mistral OCR → Chroma](/pipelines/mistral-ocr-to-chroma)

Foundations: [The Optimal Chunker for Docling](/optimal-chunker-docling) · [The Optimal Chunks for Chroma](/optimal-chunks-chroma) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)