Source: http://www.poma-ai.com/docs/pipelines/marker-to-chroma

# The Missing Link Between Marker and Optimal Retrieval in Chroma

<ByAuthor />

**The short answer:** Marker and Chroma are the natural local pair — a PDF parser that runs on your own hardware and a vector store that embeds in your process with a `PersistentClient`. Wired together naively — flatten the markdown, split, `collection.add` — the prototype works, and that's the trap: the chunking decision made at prototype time is the one that sticks. The missing link is POMA: `PrimeCut().ingest()` consumes the saved Marker JSON (the Document block tree auto-routes), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets), and the portable `.poma` archive means the same retrieval units move unchanged from your Chroma prototype to whatever runs in production.

## What each end of the pipeline actually provides

**Marker** (by Datalab / VikParuchuri) is the fast open-source PDF→markdown favourite for local pipelines. With the JSON renderer (`--output_format json`) it returns a Document block tree: a root `block_type: "Document"` with Page children, per-block HTML content, tables as full `<table>` elements with row and column spans, and a side-channel `images` dict of `{name: base64}`. What it doesn't provide: cross-page hierarchy or retrieval units. Details: [The Optimal Chunker for Marker](/optimal-chunker-marker).

**Chroma** is local-first: embedded in your process via `PersistentClient` or run client/server, with collections, `where` metadata filters and `where_document` content filters, and a default embedding function so `add` works with zero configuration — or bring your own. What it doesn't provide: any opinion about what an entry should contain. It retrieves nearest neighbors of whatever you gave it. Details: [The Optimal Chunks for the Best Retrieval in Chroma](/optimal-chunks-chroma).

## The naive wiring, and where it breaks

The common recipe — `rendered.markdown` → `RecursiveCharacterTextSplitter` → `collection.add(documents=fragments)` — breaks in a pair-specific way:

- **Non-scalar metadata blows up `add`.** Chroma metadata values must be strings, numbers, or booleans. Marker's run `metadata` object and its `images` dict of base64 bytes are neither — pipelines that pass them through fail, or strip them and lose the figures entirely.
- **`where` filters have nothing to filter on.** Marker's markdown output has no page boundaries, so `where={"page": {"$gte": 30}}` can never work — the metadata was unfillable before the first query.
- **Dangling `![](name)` refs become embedded noise.** The default embedding function faithfully embeds image-file names that point at bytes left behind in the side channel.
- **The prototype's fragments become the product's fragments.** Chroma makes it effortless to ship the first thing that works — and splitter fragments wired directly to Chroma's API have no existence outside that collection. When production wants Qdrant or pgvector, there is nothing coherent to migrate.

## The pipeline, end to end

```bash
pip install marker-pdf poma chromadb
marker_single contract.pdf --output_format json --output_dir out/
```

```python
import chromadb
from poma import PrimeCut

# 1. The missing link — raw Marker JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("out/contract/contract.json")  # Document block tree auto-detected

# 2. Chroma — embedded, persistent, zero infra. The default embedding
#    function embeds the documents locally on add.
client = chromadb.PersistentClient(path="./chroma")
collection = client.get_or_create_collection(
    name="marker_chunksets",
    metadata={"hnsw:space": "cosine"},
)

# 3. One add call: chunkset text as the document, flat scalar metadata.
leaves = [cs.chunks[-1] for cs in result.chunksets]  # leaf chunk per chunkset
collection.add(
    ids=[f"{leaf.file_id}:{leaf.chunk_index}" for leaf in leaves],
    documents=[cs.to_embed for cs in result.chunksets],
    metadatas=[
        {
            "file_id": leaf.file_id,
            "page": leaf.page,
            "depth": leaf.depth,
            "chunk_index": leaf.chunk_index,
        }
        for leaf in leaves
    ],
)

# 4. Retrieve — metadata-filtered, fully local.
hits = collection.query(
    query_texts=["What are the early termination conditions?"],
    n_results=5,
    where={"$and": [
        {"file_id": {"$eq": "contract"}},
        {"depth": {"$gte": 2}},
    ]},
)
```

Deduplicate and merge the retrieved chunksets into a cheatsheet — one prompt-ready context block — before handing them to the LLM. POMA validates the upload shape up front (a corrupted or mislabeled payload 422s immediately), splices Marker's side-channel image bytes into described text, strips running headers and footers, and rebuilds the cross-page heading tree before chunking. Bare markdown shapes (`{markdown, images, metadata}`) are not auto-routed — declare `external_ocr_source="marker"`, or `"none"` to opt out of detection.

The payoff, measured on our reference legal-document benchmark: **337 tokens** of retrieved context instead of **1,542** for a recursive character splitter, zero information loss. Methodology: [Document Ingestion & Chunking for RAG](/document-ingestion-chunking-rag).

## Metadata mapping: POMA fields → Chroma primitives

| POMA chunk field | Chroma primitive | What it enables |
| --- | --- | --- |
| `to_embed` | `document` (embedded by the collection's embedding function) | retrieval over self-explanatory chunkset text |
| `file_id` | metadata string, `where {"$eq": …}` | scope queries to one document |
| `page` (from Marker's Page blocks) | metadata int, `where {"$gte"/"$lte": …}` | page-cited answers, page-range filters |
| `depth` | metadata int, `where` filterable | filter or re-rank by hierarchy level |
| `chunk_index` | metadata int | stable ordering at assembly time |
| chunkset lineage | the document text itself + the `.poma` archive | self-contained hits, portability beyond Chroma |

## The portability angle: prototype in Chroma, keep your options

Chroma's sweet spot is the same as Marker's: everything on your machine, nothing to provision. The risk is that prototype decisions harden. Vectors are disposable — you can always re-embed — but chunks are not: they encode what your retrieval units *are*. If those units are ad-hoc splitter fragments living only inside a Chroma collection, every future migration re-opens the chunking question from scratch.

The `.poma` archive breaks that lock-in. It is the durable, database-independent record of the ingestion — chunks, chunksets, hierarchy — so the Chroma prototype and the eventual production store are two views of the same source of truth. `PrimeCut().ingest()` once; `collection.add` today; `upsert` to [Qdrant](/pipelines/marker-to-qdrant) or `INSERT` into [pgvector](/pipelines/marker-to-pgvector) tomorrow, from the same archive, with identical retrieval units.

## Frequently asked questions

### How do I get Marker output into Chroma for RAG?

Run Marker with the JSON renderer, hand the saved result to `PrimeCut().ingest()` (auto-detected, hierarchy rebuilt), then `chromadb.PersistentClient` → `get_or_create_collection` → one `collection.add` with chunkset `to_embed` as documents and `file_id`/`page`/`depth`/`chunk_index` as flat metadata. Fully local, end to end.

### Should I use Chroma's default embedding function for Marker output?

For prototyping, yes — zero configuration, runs locally. What it embeds matters more: chunkset text, not flattened markdown full of dangling image refs and cut tables. Swap in a stronger model later and re-embed the same chunksets.

### How do I filter Chroma queries by page or document for Marker-parsed PDFs?

Attach `file_id`, `page`, `depth`, `chunk_index` as metadata and use `where` filters (`$eq`, `$gte`, `$and`, …). The prerequisite: run Marker's JSON renderer, because the markdown output has no page boundaries to carry.

### Can I move a Chroma prototype built on Marker output to another vector database?

Yes — keep the `.poma` archive as the source of truth. Moving to Qdrant, pgvector, or Milvus is re-embedding and re-upserting the same chunksets; retrieval quality carries over because the retrieval units are identical.

### Why does Chroma reject my Marker metadata?

Chroma metadata must be scalar (string, number, boolean); Marker's run `metadata` object and base64 `images` dict are not. Let POMA consume them first — image bytes become described text, and the emitted chunk fields are flat scalars that map straight onto Chroma metadata.

## Related recipes

Same parser, different store: [Marker → Qdrant](/pipelines/marker-to-qdrant) · [Marker → pgvector](/pipelines/marker-to-pgvector) · [Marker → Milvus](/pipelines/marker-to-milvus)

Same store, different parser: [Docling → Chroma](/pipelines/docling-to-chroma) · [Unstructured → Chroma](/pipelines/unstructured-to-chroma) · [Mistral OCR → Chroma](/pipelines/mistral-ocr-to-chroma)

Foundations: [The Optimal Chunker for Marker](/optimal-chunker-marker) · [The Optimal Chunks for Chroma](/optimal-chunks-chroma) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)