Source: http://www.poma-ai.com/docs/optimal-chunks-chroma

# The Optimal Chunks for the Best Retrieval in Chroma

<ByAuthor />

**The short answer:** the optimal chunks for Chroma are **chunksets** — self-explanatory units that carry their full heading lineage — added as documents with hierarchy metadata (`file_id`, `page`, `depth`) that Chroma's `where` filters can act on. Chroma gets you from zero to a working retrieval loop faster than any other store; what decides whether that loop actually answers questions is what you `collection.add()` into it. And because the chunking decision you make at prototype time is the one that sticks, make it with chunks that port unchanged to whatever you run in production.

This page covers what Chroma does brilliantly, what it cannot do for you, and the chunk-and-metadata design that gets the most out of it.

## Chroma gives you the fastest path from zero to retrieval — over whatever you feed it

Chroma is the prototyping engineer's vector store, and it earns that reputation:

- **Local-first, embedded** — `chromadb.PersistentClient(path=...)` gives you a durable collection on disk with no server to run; the same API talks to a client/server deployment when you outgrow the laptop.
- **Collections** — a natural unit per corpus or experiment, created idempotently with `get_or_create_collection`.
- **`where` metadata filters + `where_document` content filters** — scope queries by any scalar metadata field, or by full-text conditions on the chunk text itself.
- **Default embedding functions, or bring your own** — pass `documents=` and Chroma embeds them for you; swap in any embedding function when you want control.

None of these primitives, however, know anything about your documents. Chroma retrieves the nearest neighbors of what you embedded. If what you embedded is a context-free fragment — a paragraph cut loose from the section that gives it meaning — Chroma will faithfully retrieve context-free fragments, on your laptop and in production alike. **The ceiling on retrieval quality is set before the first `collection.add()`.**

## The chunking decision made at prototype time is the one that sticks

Here is the pattern that plays out in almost every RAG project: someone builds a Chroma prototype in an afternoon with a quick fixed-size splitter, the demo works well enough to get funded, and then the splitter — the least examined line in the notebook — becomes load-bearing. Re-chunking later means re-embedding everything, re-running every evaluation, and explaining why retrieval behavior shifted. So nobody does it. The prototype's chunking ships.

That makes the prototype exactly the wrong place to cut corners on chunking, and exactly the right place to fix it once:

1. **Chunk with hierarchy from day one.** POMA's PrimeCut turns a PDF, DOCX, HTML file — or a bring-your-own-OCR result JSON — into **chunksets**: root-to-leaf paths through the document's heading hierarchy, each one self-explanatory when retrieved alone. See [POMA chunksets](/learn/chunking/chunksets).
2. **Keep the source of truth outside the store.** Every ingest produces a portable `.poma` archive containing the chunks, chunksets, and metadata. Chroma holds a projection of it, not the original. When production lands on Qdrant, Pinecone, or pgvector, you re-upsert the same chunksets — the chunking decision, and every evaluation you ran against it, ports unchanged.
3. **Validate on honest chunks.** A prototype evaluated on fragments tells you how well fragments retrieve. A prototype evaluated on chunksets tells you how the production system will behave.

## What "optimal chunks" means for Chroma, concretely

Four properties separate a Chroma collection that answers questions from one that returns trivia:

1. **Every document is self-explanatory.** A chunkset reads like `Master Services Agreement → Termination clauses → Early termination → "Either party may…"`. Retrieved alone, it tells the LLM (and the human reading citations) exactly where it sits. A bare 512-token window does not.
2. **No overlap, no near-duplicates.** Overlap embeds every overlapped span twice, and near-identical vectors crowd out diverse candidates in `n_results`. Chunksets carry context structurally, so overlap is unnecessary and your top-5 returns five *different* passages.
3. **Hierarchy as scalar metadata.** Chroma metadata values are scalars — which is precisely the shape of `file_id`, `page`, `depth`, and `chunk_index`. Written at ingest time, they turn `where` filters into document-aware operations: scope a query to one contract, exclude appendices by depth, cite pages in the answer. `where_document` adds content-level conditions on top.
4. **Portable by construction.** The `.poma` archive is the source of truth; Chroma, Qdrant, Pinecone, Weaviate, pgvector, and Milvus are interchangeable projections of it. Your prototype and your production system retrieve the same units.

## From document to a queryable Chroma collection

No special integration needed — chunksets map directly onto Chroma's native API:

```bash
pip install poma chromadb
```

```python
import chromadb

from poma import PrimeCut

# 1. Chunk any document — PDF, DOCX, HTML, or a bring-your-own-OCR result JSON.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.pdf")

# 2. A durable local collection — same API embedded or client/server.
chroma = chromadb.PersistentClient(path="./chroma")
collection = chroma.get_or_create_collection("contracts")

# 3. Add chunksets: Chroma embeds `documents` with its default embedding
#    function (or bring your own); hierarchy lands as filterable metadata.
collection.add(
    documents=[cs.to_embed for cs in result.chunksets],
    metadatas=[
        {"file_id": cs.file_id, "page": cs.page, "depth": cs.depth}
        for cs in result.chunksets
    ],
    ids=[f"{cs.file_id}-{i}" for i, cs in enumerate(result.chunksets)],
)

# 4. Query, scoped to one document via a where filter.
hits = collection.query(
    query_texts=["What does the contract say about early termination?"],
    n_results=5,
    where={"file_id": result.chunksets[0].file_id},
)
```

After retrieval, merge the hits instead of concatenating them: chunksets from the same section share ancestor headings, and deduplicating that shared lineage assembles one coherent context block — a **cheatsheet**. The `.poma` archive (or your document store) holds everything needed to rebuild it from the retrieved IDs. On our reference legal-document benchmark, cheatsheet assembly answered with **337 tokens** of context versus **1,542** for a recursive-splitter baseline, with zero information loss — [methodology here](/document-ingestion-chunking-rag).

## Chunk shapes in Chroma, compared

| Property | Fixed 512-token chunks + overlap | Semantic chunks | **POMA chunksets** |
| --- | --- | --- | --- |
| Document is self-explanatory when retrieved alone | ✗ | Sometimes | **✓ (by construction)** |
| Near-duplicate vectors in the collection | Many (overlap) | Few | **None** |
| Scalar metadata for `where` filters | Manual | Manual | **✓ (`file_id`, `page`, `depth`)** |
| Tables survive embedding | ✗ (cut mid-row) | Fragile | **✓ (HTML, never cut)** |
| Ports from prototype to production DB unchanged | Re-chunk + re-eval | Re-chunk + re-eval | **✓ (`.poma` archive)** |
| Post-retrieval assembly | Concatenate raw hits | Concatenate raw hits | **Cheatsheets (deduped lineage)** |

## Frequently asked questions

### What is the optimal chunk size for ChromaDB?

There is no universal token count — the optimal unit is a chunk that is self-explanatory without its neighbors. Chunksets achieve that at any size, which is why they beat fixed 512-token windows: every document in the collection can be retrieved alone and still be understood. If you must fix a size, start around 512 tokens — but the size knob cannot restore missing context.

### What metadata should I store with chunks in ChromaDB?

`file_id`, `page`, `depth`, `chunk_index` — flat scalar fields, which is exactly what Chroma metadata requires (strings, ints, floats, booleans). Every one of them is filterable with `where`, enabling per-document scoping, page citations, and depth-aware filtering.

### How do I filter chunks by metadata in ChromaDB?

Pass a `where` clause to `collection.query` — `where={"file_id": "contract-2024"}` scopes to one document; `$in`, `$gte`, `$ne` compose richer filters. `where_document` adds full-text conditions on the chunk content (`$contains`). Both depend on metadata written at ingest time.

### Does chunk overlap hurt a Chroma collection?

Yes. Every overlapped span is embedded twice, and the resulting near-duplicates crowd out diverse candidates in `n_results`. Chunks that carry their heading lineage structurally don't need overlap at all.

### Can I move my Chroma prototype to another vector database without re-chunking?

Yes — if the chunks live outside Chroma as a portable artifact. The `.poma` archive is the source of truth; moving to Qdrant, Pinecone, Weaviate, pgvector, or Milvus is a re-upsert of the same chunksets with the same metadata, not a re-chunking project.

### How do I turn Chroma query results into prompt-ready context?

Merge, don't concatenate: retrieved chunksets share ancestor headings, so deduplicate the shared lineage into one coherent block — a cheatsheet. That assembly is how the reference benchmark answers with 337 tokens instead of 1,542.

## First-party ingest: vektoria's push()

Chroma is one of only two databases (with pgvector) that has a real first-party `vektoria` ingest connector — one call handles chunking-to-vector in a content-free shape:

```python
from poma.vektoria import push, get_connector
from poma.embeddings import get_embedder

stats = push(
    "contract.poma",
    volume="s3://your-bucket/poma",
    connector=get_connector("chroma", collection="poma", path="./chroma"),
    embedder=get_embedder("local:BAAI/bge-small-en-v1.5"),
)
```

`push()` writes content to the volume first (so a vector id never points at missing content), then embeds and upserts content-free vectors — `(id, vector, {file_id, chunkset_index})` — via the connector. Retrieval is unchanged: run your normal `collection.query(...)`, then `assemble(results, volume=VOL)`.

## Feed Chroma from the parser you already run

Pair this page with your ingestion side — the same chunksets, straight from your OCR/parsing tool's raw output:

- [Mistral OCR → Chroma](/pipelines/mistral-ocr-to-chroma)
- [LlamaParse → Chroma](/pipelines/llamaparse-to-chroma)
- [Azure Document Intelligence → Chroma](/pipelines/azure-document-intelligence-to-chroma)
- [Unstructured.io → Chroma](/pipelines/unstructured-to-chroma)
- [Docling → Chroma](/pipelines/docling-to-chroma)
- [Marker → Chroma](/pipelines/marker-to-chroma)
- [AWS Textract → Chroma](/pipelines/textract-to-chroma)

Graduating to a different store? The same chunk design applies: [Qdrant](/optimal-chunks-qdrant) · [Pinecone](/optimal-chunks-pinecone) · [Weaviate](/optimal-chunks-weaviate) · [pgvector](/optimal-chunks-pgvector) · [Milvus](/optimal-chunks-milvus).

Fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [RAG architecture guide](/guides/rag-architecture/) · [POMA chunksets](/learn/chunking/chunksets).