AWS Textract to Chroma: Chunking for RAG
The short answer: AWS Textract gives you a LAYOUT-annotated block graph; Chroma gives you the fastest path from zero to working retrieval — an embedded, local-first store with metadata filters. Wired together naively — flatten Blocks[], split, add — they produce a prototype that lies to you: fine on single-column demo files, scrambled on the multi-column documents Textract is actually for. The missing link is POMA: PrimeCut().ingest() consumes the raw AnalyzeDocument JSON (auto-detected), emits hierarchy-preserving chunksets, and those go into a PersistentClient collection with hierarchy metadata for where-filters — while the portable .poma archive keeps the same chunks valid for whatever database the prototype graduates to.
What each end of the pipeline actually provides
AWS Textract (AnalyzeDocument with FeatureTypes: ["LAYOUT", "TABLES"]) returns a flat Blocks[] graph: LAYOUT_* blocks in multi-column-aware reading order, LAYOUT_TITLE and LAYOUT_SECTION_HEADER marking structure, structured TABLE grids with MERGED_CELL spans. What it doesn't provide: markdown, cross-page hierarchy, or retrieval units — and without the LAYOUT feature, not even reliable reading order. Details: The Optimal Chunker for AWS Textract.
Chroma runs embedded in your process or as a lightweight server, persists to a local directory via PersistentClient, embeds added documents with a default embedding function (or one you bring), and filters queries with where (metadata) and where_document (content) clauses. What it doesn't provide: any opinion about what a document entry should contain. It retrieves nearest neighbors of whatever you added — including scrambled Block fragments, if that's what the prototype glue produced. Details: The Optimal Chunks for the Best Retrieval in Chroma.
The naive wiring, and where it breaks
The common recipe — iterate LINE blocks, join with newlines, RecursiveCharacterTextSplitter, collection.add(...) — breaks in a pair-specific way:
- The prototype passes on the wrong corpus. Position-sorting
LINEblocks happens to work on the clean, single-column memo used for the demo. On the two-column reports and forms Textract is bought for, it interleaves the columns into scrambled prose — and Chroma's default embedding function embeds it without complaint. Nothing errors; retrieval just quietly degrades in production. This is the most dangerous failure mode a prototyping store can have: false confidence. - The chunking decision sticks. Chroma's whole point is speed to first result, so the improvised flatten-and-split lands in glue code — and the chunking decision made at prototype time is the one that ships. When the project graduates to a hosted database, the units either port as-is or get re-invented, invalidating every retrieval evaluation the prototype produced.
- Metadata never exists. Flattening destroys Textract's page numbers and heading roles before
add()is called, sowhere={"page": ...}and depth filters — Chroma's retrieval primitives — have nothing to work with.
The pipeline, end to end
pip install boto3 poma chromadbimport json
import os
import boto3
import chromadb
from poma import PrimeCut
from poma.utils import unpack_poma_archive
# 1. Textract — your existing call, unchanged. LAYOUT is required.
textract = boto3.client("textract")
with open("contract.png", "rb") as f:
response = textract.analyze_document(
Document={"Bytes": f.read()},
FeatureTypes=["LAYOUT", "TABLES"],
)
with open("contract.textract.json", "w") as f:
json.dump(response, f)
# 2. The missing link — raw Blocks[] in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.textract.json", # Textract shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive — page numbers live in its chunk records
)
# A chunkset's structural metadata comes from its leaf chunk (cs.chunks is in
# document order, so the last index is the leaf). depth is on the SDK objects;
# page only exists on the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
c["chunk_index"]: c.get("page")
for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}
# 3. Chroma — persistent local collection, hierarchy in the metadata.
chroma = chromadb.PersistentClient(path="./chroma-data")
collection = chroma.get_or_create_collection(
name="contracts",
metadata={"hnsw:space": "cosine"},
)
collection.add(
ids=[f"{cs.file_id}:{cs.chunkset_index}" for cs in result.chunksets],
documents=[cs.to_embed for cs in result.chunksets], # embedded by the
# collection's embedding function — default model or bring your own
metadatas=[
{
"file_id": cs.file_id,
"chunkset_index": cs.chunkset_index,
"page": page_of[cs.chunks[-1]],
"depth": depth_of[cs.chunks[-1]],
}
for cs in result.chunksets
],
)
# 4. Filtered query — scope to one document, cite pages from metadata.
hits = collection.query(
query_texts=["What are the early termination conditions?"],
n_results=3,
where={"file_id": result.chunksets[0].file_id},
)POMA validates the payload up front: a Textract result missing LAYOUT blocks fails with a clear 422 telling you to re-run with FeatureTypes including "LAYOUT" — the scramble never reaches your collection. Tables are spliced by geometry and arrive as HTML with rowspan/colspan; figures (Textract returns no crops) are counted as offloaded content in content_metadata. To force or suppress detection, set external_ocr_source to "textract" or "none".
Deduplicate and merge the retrieved chunksets in chunkset_index order and you have a cheatsheet — one prompt-ready context block. On our reference legal document, that assembly answers the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss. Methodology: Document Ingestion & Chunking for RAG.
Prototype honestly, port cleanly
Chroma is where chunking decisions get made — usually implicitly, inside glue code, on unrepresentative sample files. Two habits keep the prototype honest:
- Chunk outside the store. The
.pomaarchive — chunks and chunksets derived from your Textract results — is the portable source of truth. The Chroma collection is a disposable projection of it: rebuild locally, hand a teammate the archive, or load the identical units into the production database later. Retrieval evaluations done on the prototype stay valid because the units don't change, only the store does. - Prototype on the hard documents. Two-column layouts, merged-cell tables, checkbox forms — the inputs that justify Textract in the first place. POMA's LAYOUT-spine reading order and geometry-spliced HTML tables behave identically at prototype scale and production scale, so what you measure locally is what you ship.
Metadata mapping: POMA fields → Chroma primitives
| POMA chunk field | Chroma primitive | What it enables |
|---|---|---|
to_embed | documents entry, embedded by the collection's embedding function | semantic retrieval; where_document content filters |
file_id | metadata key (string), where={"file_id": ...} | scope queries to one document |
page (on the archive's chunk records; take the chunkset's leaf chunk) | metadata key (int), where with $gte/$lte | page-cited answers, page-range filters |
depth of the leaf chunk | metadata key (int) | filter or re-rank by hierarchy level |
chunkset_index | metadata key (int) | stable ordering at assembly time |
| chunkset lineage | root-to-leaf text inside the document entry; .poma archive alongside | self-explanatory hits, portability to any other store |
Frequently asked questions
How do I get AWS Textract results into Chroma for RAG?
Save the raw AnalyzeDocument JSON (with LAYOUT in the feature list), run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), then collection.add() each chunkset's to_embed as a document with file_id/chunkset_index/page/depth metadata on a PersistentClient collection. Query with where-filters and assemble cheatsheets.
What metadata should Textract chunks carry in a Chroma collection?
file_id (string), chunkset_index (int), page (int), and depth (int) — all scalar, as Chroma requires. file_id, chunkset_index, and depth come off the SDK's chunkset and chunk objects; page comes from the .poma archive's chunk records, read from the chunkset's leaf chunk. They enable document scoping, page-range filters via $gte/$lte, hierarchy filtering, and stable assembly order.
Why does my Textract-to-Chroma prototype work on memos but fail on two-column PDFs?
Position-sorting LINE blocks works on single-column memos and silently interleaves multi-column layouts. Chroma embeds the scrambled prose without complaint, so nothing errors — retrieval just degrades. POMA follows the LAYOUT block sequence instead and 422s on payloads without LAYOUT.
Can I move a Chroma prototype built on Textract output to another vector database later?
Yes — the .poma archive is the portable source of truth. The identical chunks and chunksets load into Qdrant, pgvector, Milvus, or any other store, so prototype retrieval evaluations stay valid after the move.
Should I use Chroma's default embedding function for Textract chunksets?
For prototyping, yes — zero wiring. Chunksets make even a small default model useful because to_embed is normalized and self-explanatory. Swapping in your own embedding function later changes nothing about the units or metadata.
Related recipes
Same parser, different store: Textract → Qdrant · Textract → pgvector · Textract → Milvus
Same store, different parser: Mistral OCR → Chroma · Docling → Chroma · LlamaParse → Chroma
Foundations: The Optimal Chunker for AWS Textract · The Optimal Chunks for Chroma · All pipeline recipes · RAG chunking guide