Unstructured.io to Chroma: Chunking for RAG
The short answer: Unstructured.io gives you a typed, ordered element list; Chroma gives you the fastest path from zero to a working retrieval loop — an embedded, persistent collection with where filters and batteries-included embeddings. Wired together naively — element texts as documents, element IDs as IDs — the demo works in five minutes and fossilizes flat fragments into your product. The missing link is POMA: PrimeCut().ingest() consumes the raw elements_to_json output (auto-detected), emits hierarchy-preserving chunksets, and those become Chroma documents with flat, where-filterable metadata — and because the .poma archive is the portable source of truth, the same chunks move to any production store unchanged.
What each end of the pipeline actually provides
Unstructured.io (the open-source unstructured library and the hosted API) returns a flat, ordered list of typed elements — Title, NarrativeText, ListItem, Table, Image — each with its text, a stable element_id, and metadata such as page_number, text_as_html for tables, and optionally image_base64. What it doesn't provide: hierarchy (the list is flat — nothing records which Title nests under which) or retrieval units; the built-in by_title and basic strategies emit flat fragments. Details: The Optimal Chunker for Unstructured.io.
Chroma provides a local-first vector store that runs embedded or client/server behind the same API: persistent collections via PersistentClient, metadata filtering with where and content filtering with where_document, and default embedding functions so add(documents=...) just works. What it doesn't provide: any opinion about what a document should be, or nested metadata — values must be flat scalars. It retrieves nearest neighbors of whatever you added, fragments included. Details: The Optimal Chunks for the Best Retrieval in Chroma.
The naive wiring, and where it breaks
Unstructured and Chroma line up so neatly that the naive wiring is one line — collection.add(documents=[el.text for el in elements], ids=[el.id for el in elements]) — and that is exactly the problem:
- Chroma rejects the element metadata, so people drop it. Unstructured's
metadatais a nested dict; Chroma metadata must be flat scalars (str,int,float,bool). Theadd()raises a validation error, the quickest fix is passing nometadatasat all — and with it goes everywherefilter: no per-document scoping, no page citations, nothing. - The five-minute demo fossilizes. Per-element documents look fine on toy queries against one PDF. Then the prototype ships, and top-k at production scale is a pile of orphan fragments — three-word
Titledocuments, flattened tables, recurringHeader/Footertext — because the chunking decision made on day one is the one nobody revisits. - Prototype-to-production means re-chunking. When the team migrates off the prototype store, per-element units have to be re-thought from scratch, and every retrieval eval resets to zero.
POMA's chunksets fix this at the source: self-explanatory root-to-leaf units, furniture dropped, tables spliced whole, hierarchy metadata as flat scalars Chroma accepts — and a portable archive so the units outlive the prototype. On a notoriously hard reference legal document, chunksets plus cheatsheet assembly delivered the answer in 337 tokens of retrieved context versus 1,542 for a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install 'unstructured[pdf]' poma chromadbimport chromadb
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from unstructured.partition.pdf import partition_pdf
from unstructured.staging.base import elements_to_json
# 1. Unstructured — your existing call, unchanged.
elements = partition_pdf(
filename="contract.pdf",
strategy="hi_res",
infer_table_structure=True, # populates metadata.text_as_html
extract_image_block_types=["Image"],
extract_image_block_to_payload=True, # inline base64 → POMA describes figures
)
elements_to_json(elements, filename="contract.unstructured.json")
# 2. The missing link — raw element list in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest( # Unstructured shape auto-detected
"contract.unstructured.json",
download_dir="store",
filename="contract.poma", # keep the archive — page numbers live in it
)
# Depth comes off the SDK objects; page is on the archive's chunk records only.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
c["chunk_index"]: c.get("page")
for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}
# 3. Chroma — a durable local collection; the default embedding
# function embeds `documents`, or bring your own.
chroma = chromadb.PersistentClient(path="./chroma")
collection = chroma.get_or_create_collection("contracts")
leaves = [cs.chunks[-1] for cs in result.chunksets] # document order — last is the leaf
collection.add(
documents=[cs.to_embed for cs in result.chunksets],
metadatas=[
{"file_id": cs.file_id, "chunkset_index": cs.chunkset_index,
"page": page_of[leaf], "depth": depth_of[leaf]}
for cs, leaf in zip(result.chunksets, leaves)
],
ids=[f"{cs.file_id}:{cs.chunkset_index}" for cs in result.chunksets],
)
# 4. Query with a where filter, then assemble the cheatsheet.
hits = collection.query(
query_texts=["What are the early termination conditions?"],
n_results=5,
where={"file_id": result.chunksets[0].file_id},
)
ordered = sorted(
zip(hits["documents"][0], hits["metadatas"][0]),
key=lambda pair: pair[1]["chunkset_index"],
)
cheatsheet = "\n\n".join(dict.fromkeys(doc for doc, _ in ordered))POMA validates the payload shape up front (fingerprint: elements carrying type + element_id; a corrupted or mislabeled upload 422s immediately), groups elements by page_number, splices each Table's text_as_html, describes inline image_base64 figures (disk-bound image_path refs are neutralized and counted in content_metadata), drops Header/Footer/PageNumber furniture, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "unstructured" or "none".
When the prototype graduates, nothing is re-parsed and nothing is re-chunked: the .poma archive holds the same chunks and chunksets you loaded here, ready to re-upsert into Qdrant, pgvector, or Milvus. The retrieval units — and every eval you ran against them — carry over; only the metadata mapping changes.
Metadata mapping: POMA fields → Chroma primitives
| POMA chunk field | Chroma primitive | What it enables |
|---|---|---|
to_embed | document (default embedding function or your own) | paraphrase retrieval; where_document content matching |
file_id | scalar metadata, where={"file_id": ...} | scope queries to one document |
page of the chunkset's leaf chunk (from the archive's chunk records, filled from Unstructured metadata.page_number) | scalar metadata, where={"page": {"$lte": N}} | page-cited answers, page-range scoping |
depth of the leaf chunk | scalar metadata | filter by hierarchy level |
chunkset_index | scalar metadata, id construction ({file_id}:{chunkset_index}) | stable ordering at cheatsheet assembly |
| chunkset text + lineage | document body + .poma archive | assembly now, portability later |
Unstructured's nested metadata dict | not metadata — scalars only | avoids Chroma's flat-scalar validation error |
Frequently asked questions
How do I get Unstructured.io elements into Chroma for RAG?
Save the element list with elements_to_json, run PrimeCut().ingest() on the raw JSON (auto-detected, hierarchy rebuilt), then collection.add() the chunksets: to_embed as documents, file_id/chunkset_index/page/depth as flat scalar metadata — page and depth taken from the chunkset's leaf chunk, with page read out of the .poma archive's chunk records. Query with a where filter and merge hits into a cheatsheet.
Why does Chroma reject Unstructured element metadata on add()?
Chroma metadata must be flat scalars, and Unstructured's metadata is a nested dict — so add() raises a validation error, and dropping metadata entirely (the common workaround) silently costs every where filter. Map deliberately: four scalars in metadata, bulky values in the document body.
Can Chroma where filters use Unstructured's page numbers?
Yes — POMA carries metadata.page_number through as a plain int on every chunk record in the .poma archive; read it back with unpack_poma_archive, write the chunkset's leaf-chunk page into Chroma metadata, and where={"page": {"$lte": 10}} works alongside file_id scoping and depth filters. None of it works if the page number died in a flatten-and-split step.
Will a Chroma prototype built on Unstructured output port to production?
With POMA, yes: the .poma archive is the portable source of truth, so the same chunks and chunksets re-upsert into Qdrant, pgvector, or Milvus without re-parsing or re-chunking. Only the metadata mapping changes.
Should I use Chroma's default embedding function for POMA chunksets?
For prototyping, yes — and because each chunkset is self-explanatory, even a small default model retrieves better than fragment embeddings from a larger one. Swap in your own embedding function for production; the chunksets don't change.
Related recipes
Same parser, different store: Unstructured.io → Qdrant · Unstructured.io → pgvector · Unstructured.io → Milvus
Same store, different parser: Mistral OCR → Chroma · Docling → Chroma · Marker → Chroma
Foundations: The Optimal Chunker for Unstructured.io · The Optimal Chunks for Chroma · All pipeline recipes · RAG chunking guide