AWS Textract to MongoDB Atlas: Chunking for RAG
The short answer: AWS Textract gives you an excellent, forms-and-tables-aware layout graph; MongoDB Atlas gives you vector search folded directly into the aggregation pipeline you already run. Wired together naively — flatten Blocks[], split, embed — they still produce mediocre RAG, because nothing in between rebuilds Textract's own reading order or projects the metadata Atlas needs at query time. The missing link is POMA: PrimeCut().ingest() consumes the raw AnalyzeDocument response, follows the LAYOUT spine, and emits hierarchy-preserving chunksets; vektoria's records_from_archive and assemble() keep the collection content-free and turn a properly-projected $vectorSearch result into prompt-ready context.
What each end of the pipeline actually provides
AWS Textract (AnalyzeDocument with the LAYOUT feature) returns a flat Blocks[] graph: LAYOUT_TITLE/LAYOUT_SECTION_HEADER blocks in multi-column-aware reading order, structured TABLE blocks with no Id link to their LAYOUT_TABLE region, and LAYOUT_FIGURE regions with no image bytes at all. What it doesn't provide: cross-page hierarchy, a table-to-layout link, or any retrieval unit whatsoever. Details: The Optimal Chunker for AWS Textract.
MongoDB Atlas provides $vectorSearch as an aggregation stage, compound hybrid search via Atlas Search, and ordinary document fields you can $match alongside the vector query — all inside the cluster you already operate. What it doesn't provide: any field in the aggregation output you didn't explicitly project. Details: MongoDB Atlas Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — flatten Blocks[] to text, split into fixed windows, insert_one per chunk — breaks in a pair-specific way:
- Flatten
Blocks[]without honoringLAYOUTand Textract's multi-column reports interleave into scrambled prose before they ever reach Mongo — no aggregation stage fixes reading order after the fact. - Insert one document per fixed-size window with no metadata field, and
file_id/pagescoping is impossible without a full collection scan — Atlas ranks whatever you wrote, nothing more. - Even correctly-chunked text fails silently at the query layer. A
$vectorSearchstage with no follow-up$projectof the metadata field returns a non-empty hit list that POMA'sassemble()cannot readfile_idorchunkset_indexfrom — it comes back an empty list, indistinguishable from "no relevant chunks" unless you already know to check the aggregation pipeline.
The pipeline, end to end
pip install boto3 pymongoimport json
import os
import boto3
from pymongo import MongoClient
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
# 1. Your existing Textract call — LAYOUT is required; add TABLES for cell grids.
textract = boto3.client("textract")
with open("contract.png", "rb") as f:
response = textract.analyze_document(
Document={"Bytes": f.read()},
FeatureTypes=["LAYOUT", "TABLES"],
)
with open("contract.textract.json", "w") as f:
json.dump(response, f)
# 2. The missing link — raw Textract result in, a portable .poma archive out.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.textract.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
mongo = MongoClient(os.environ["MONGODB_ATLAS_URI"])
col = mongo["poma"]["chunksets"]
# Atlas vector index (numDimensions, similarity) is created once via the Atlas UI/API,
# named "poma_vec" here — not through the pymongo driver itself.
# 3. Content-free ingest — MongoDB holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
for rec in records:
vol.write_doc(rec.payload["file_id"], {"file_id": rec.payload["file_id"],
"chunks": rec.payload["chunks"], "text": rec.text})
col.insert_one({
"_id": rec.id,
"embedding": embedder.embed([rec.text])[0],
"metadata": {"file_id": rec.payload["file_id"], "chunkset_index": rec.payload["chunkset_index"]},
})
# 4. $vectorSearch retrieval — $project the metadata explicitly, then assemble.
qv = embedder.embed(["early termination conditions"])[0]
hits = list(col.aggregate([
{"$vectorSearch": {"index": "poma_vec", "path": "embedding",
"queryVector": qv, "numCandidates": 200, "limit": 10}},
{"$project": {"_id": 1, "metadata": 1, "score": {"$meta": "vectorSearchScore"}}},
]))
context = assemble(hits, volume=vol) # -> [{"file_id", "content"}, ...]Textract's own quirks are handled before a single document reaches Atlas: TABLE blocks are spliced into their LAYOUT_TABLE region by bounding-box geometry and rendered as HTML with rowspan/colspan, SELECTION_ELEMENT checkboxes survive as ☒/☐, and offloaded figures are counted, never silently dropped. A payload missing LAYOUT blocks 422s up front rather than shipping scrambled reading order.
Metadata mapping: POMA fields → MongoDB Atlas primitives
| POMA chunk field | MongoDB Atlas primitive | What it enables |
|---|---|---|
to_embed | embedding field, covered by the Atlas Vector Search index | $vectorSearch ANN |
file_id | metadata.file_id document field | $match per-document scoping |
chunkset_index | metadata.chunkset_index document field | assemble() dedup id, deterministic routing |
page (Textract layout-derived) | metadata.page document field | page-cited answers, $match range filters |
depth | metadata.depth document field | hierarchy-aware filter/re-rank |
| chunkset content | volume (Volume.write_doc), not in Mongo | content-free documents, cheatsheet assembly |
Frequently asked questions
How do I get AWS Textract results into MongoDB Atlas for RAG?
Run AnalyzeDocument with the LAYOUT feature as you already do, save the raw JSON, and hand it to PrimeCut, which auto-detects the Textract shape and rebuilds the reading order into chunks and chunksets. records_from_archive gives you deterministic ids and embeddable text to insert as documents covered by an Atlas Vector Search index.
Why does POMA's assemble() return nothing from my MongoDB Atlas Textract index?
Atlas's one documented gotcha: $vectorSearch must be followed by a $project that explicitly includes the metadata field. Skip it and a real hit list comes back with a score but no file_id/chunkset_index — assemble() returns an empty list.
How are Textract's spliced tables represented once they land in MongoDB Atlas?
Textract has no Id link between a LAYOUT_TABLE region and its TABLE block, so POMA splices them by geometry before chunking — the cell grid arrives as HTML with rowspan/colspan already inline in the chunkset text stored in the collection.
What Atlas document fields should AWS Textract chunks carry?
An embedding field covered by the vector index, plus a metadata object with file_id, chunkset_index, page, and depth — one object keeps the required $project a single inclusion.
Should I switch from Textract to another OCR to fix MongoDB Atlas retrieval quality?
Usually not. Atlas ranks whatever you indexed; weak retrieval is almost always upstream, in how Blocks becomes chunks. A BYOCR chunker closes that gap without touching your Textract contract or your cluster.
Related recipes
Same parser, different store: Textract → Redis · Textract → LanceDB · Textract → Qdrant
Foundations: The Optimal Chunker for AWS Textract · MongoDB Atlas Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide