Skip to content

Unstructured.io to Chroma: Chunking for RAG ​

The short answer: Unstructured.io gives you a typed, ordered element list; Chroma gives you the fastest path from zero to a working retrieval loop — an embedded, persistent collection with where filters and batteries-included embeddings. Wired together naively — element texts as documents, element IDs as IDs — the demo works in five minutes and fossilizes flat fragments into your product. The missing link is POMA: PrimeCut().ingest() consumes the raw elements_to_json output (auto-detected), emits hierarchy-preserving chunksets, and those become Chroma documents with flat, where-filterable metadata — and because the .poma archive is the portable source of truth, the same chunks move to any production store unchanged.

What each end of the pipeline actually provides ​

Unstructured.io (the open-source unstructured library and the hosted API) returns a flat, ordered list of typed elements — Title, NarrativeText, ListItem, Table, Image — each with its text, a stable element_id, and metadata such as page_number, text_as_html for tables, and optionally image_base64. What it doesn't provide: hierarchy (the list is flat — nothing records which Title nests under which) or retrieval units; the built-in by_title and basic strategies emit flat fragments. Details: The Optimal Chunker for Unstructured.io.

Chroma provides a local-first vector store that runs embedded or client/server behind the same API: persistent collections via PersistentClient, metadata filtering with where and content filtering with where_document, and default embedding functions so add(documents=...) just works. What it doesn't provide: any opinion about what a document should be, or nested metadata — values must be flat scalars. It retrieves nearest neighbors of whatever you added, fragments included. Details: The Optimal Chunks for the Best Retrieval in Chroma.

The naive wiring, and where it breaks ​

Unstructured and Chroma line up so neatly that the naive wiring is one line — collection.add(documents=[el.text for el in elements], ids=[el.id for el in elements]) — and that is exactly the problem:

  • Chroma rejects the element metadata, so people drop it. Unstructured's metadata is a nested dict; Chroma metadata must be flat scalars (str, int, float, bool). The add() raises a validation error, the quickest fix is passing no metadatas at all — and with it goes every where filter: no per-document scoping, no page citations, nothing.
  • The five-minute demo fossilizes. Per-element documents look fine on toy queries against one PDF. Then the prototype ships, and top-k at production scale is a pile of orphan fragments — three-word Title documents, flattened tables, recurring Header/Footer text — because the chunking decision made on day one is the one nobody revisits.
  • Prototype-to-production means re-chunking. When the team migrates off the prototype store, per-element units have to be re-thought from scratch, and every retrieval eval resets to zero.

POMA's chunksets fix this at the source: self-explanatory root-to-leaf units, furniture dropped, tables spliced whole, hierarchy metadata as flat scalars Chroma accepts — and a portable archive so the units outlive the prototype. On a notoriously hard reference legal document, chunksets plus cheatsheet assembly delivered the answer in 337 tokens of retrieved context versus 1,542 for a recursive character splitter, with zero information loss (methodology).

The pipeline, end to end ​

bash
pip install 'unstructured[pdf]' poma chromadb
python
import chromadb
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from unstructured.partition.pdf import partition_pdf
from unstructured.staging.base import elements_to_json

# 1. Unstructured — your existing call, unchanged.
elements = partition_pdf(
    filename="contract.pdf",
    strategy="hi_res",
    infer_table_structure=True,           # populates metadata.text_as_html
    extract_image_block_types=["Image"],
    extract_image_block_to_payload=True,  # inline base64 → POMA describes figures
)
elements_to_json(elements, filename="contract.unstructured.json")

# 2. The missing link — raw element list in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(  # Unstructured shape auto-detected
    "contract.unstructured.json",
    download_dir="store",
    filename="contract.poma",  # keep the archive — page numbers live in it
)

# Depth comes off the SDK objects; page is on the archive's chunk records only.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
    c["chunk_index"]: c.get("page")
    for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}

# 3. Chroma — a durable local collection; the default embedding
#    function embeds `documents`, or bring your own.
chroma = chromadb.PersistentClient(path="./chroma")
collection = chroma.get_or_create_collection("contracts")

leaves = [cs.chunks[-1] for cs in result.chunksets]  # document order — last is the leaf
collection.add(
    documents=[cs.to_embed for cs in result.chunksets],
    metadatas=[
        {"file_id": cs.file_id, "chunkset_index": cs.chunkset_index,
         "page": page_of[leaf], "depth": depth_of[leaf]}
        for cs, leaf in zip(result.chunksets, leaves)
    ],
    ids=[f"{cs.file_id}:{cs.chunkset_index}" for cs in result.chunksets],
)

# 4. Query with a where filter, then assemble the cheatsheet.
hits = collection.query(
    query_texts=["What are the early termination conditions?"],
    n_results=5,
    where={"file_id": result.chunksets[0].file_id},
)
ordered = sorted(
    zip(hits["documents"][0], hits["metadatas"][0]),
    key=lambda pair: pair[1]["chunkset_index"],
)
cheatsheet = "\n\n".join(dict.fromkeys(doc for doc, _ in ordered))

POMA validates the payload shape up front (fingerprint: elements carrying type + element_id; a corrupted or mislabeled upload 422s immediately), groups elements by page_number, splices each Table's text_as_html, describes inline image_base64 figures (disk-bound image_path refs are neutralized and counted in content_metadata), drops Header/Footer/PageNumber furniture, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "unstructured" or "none".

When the prototype graduates, nothing is re-parsed and nothing is re-chunked: the .poma archive holds the same chunks and chunksets you loaded here, ready to re-upsert into Qdrant, pgvector, or Milvus. The retrieval units — and every eval you ran against them — carry over; only the metadata mapping changes.

Metadata mapping: POMA fields → Chroma primitives ​

POMA chunk fieldChroma primitiveWhat it enables
to_embeddocument (default embedding function or your own)paraphrase retrieval; where_document content matching
file_idscalar metadata, where={"file_id": ...}scope queries to one document
page of the chunkset's leaf chunk (from the archive's chunk records, filled from Unstructured metadata.page_number)scalar metadata, where={"page": {"$lte": N}}page-cited answers, page-range scoping
depth of the leaf chunkscalar metadatafilter by hierarchy level
chunkset_indexscalar metadata, id construction ({file_id}:{chunkset_index})stable ordering at cheatsheet assembly
chunkset text + lineagedocument body + .poma archiveassembly now, portability later
Unstructured's nested metadata dictnot metadata — scalars onlyavoids Chroma's flat-scalar validation error

Frequently asked questions ​

How do I get Unstructured.io elements into Chroma for RAG? ​

Save the element list with elements_to_json, run PrimeCut().ingest() on the raw JSON (auto-detected, hierarchy rebuilt), then collection.add() the chunksets: to_embed as documents, file_id/chunkset_index/page/depth as flat scalar metadata — page and depth taken from the chunkset's leaf chunk, with page read out of the .poma archive's chunk records. Query with a where filter and merge hits into a cheatsheet.

Why does Chroma reject Unstructured element metadata on add()? ​

Chroma metadata must be flat scalars, and Unstructured's metadata is a nested dict — so add() raises a validation error, and dropping metadata entirely (the common workaround) silently costs every where filter. Map deliberately: four scalars in metadata, bulky values in the document body.

Can Chroma where filters use Unstructured's page numbers? ​

Yes — POMA carries metadata.page_number through as a plain int on every chunk record in the .poma archive; read it back with unpack_poma_archive, write the chunkset's leaf-chunk page into Chroma metadata, and where={"page": {"$lte": 10}} works alongside file_id scoping and depth filters. None of it works if the page number died in a flatten-and-split step.

Will a Chroma prototype built on Unstructured output port to production? ​

With POMA, yes: the .poma archive is the portable source of truth, so the same chunks and chunksets re-upsert into Qdrant, pgvector, or Milvus without re-parsing or re-chunking. Only the metadata mapping changes.

Should I use Chroma's default embedding function for POMA chunksets? ​

For prototyping, yes — and because each chunkset is self-explanatory, even a small default model retrieves better than fragment embeddings from a larger one. Swap in your own embedding function for production; the chunksets don't change.

Same parser, different store: Unstructured.io → Qdrant · Unstructured.io → pgvector · Unstructured.io → Milvus

Same store, different parser: Mistral OCR → Chroma · Docling → Chroma · Marker → Chroma

Foundations: The Optimal Chunker for Unstructured.io · The Optimal Chunks for Chroma · All pipeline recipes · RAG chunking guide