The Optimal Chunker for AWS Textract
The short answer: the optimal chunker for AWS Textract is one that consumes the raw AnalyzeDocument response as-is and reads the LAYOUT spine properly — following the layout block sequence for multi-column reading order, promoting LAYOUT_TITLE and LAYOUT_SECTION_HEADER to headings, and splicing structured TABLE blocks into place — instead of flattening Blocks[] to plain text. POMA's bring-your-own-OCR connector does exactly this: you upload the unmodified Textract JSON, and PrimeCut returns hierarchy-preserving chunks and chunksets — no re-OCR, no Block flattening, no hand-rolled splitter.
This page explains what Textract gives you, why the LAYOUT feature is non-negotiable, why the usual Blocks-to-text flattening squanders it, and how to go from a boto3 call to embedded chunksets in a few lines.
What AWS Textract gives you — and where it stops
Textract's AnalyzeDocument API returns a flat Blocks[] array — a graph of typed blocks linked by Ids. What's in that array depends entirely on the FeatureTypes you request:
- Baseline (always):
PAGE,LINE,WORDblocks — the recognized text with bounding-box geometry, but no reading order beyond position on the page. LAYOUT:LAYOUT_TITLE,LAYOUT_SECTION_HEADER,LAYOUT_TEXT,LAYOUT_LIST,LAYOUT_TABLE,LAYOUT_FIGURE,LAYOUT_HEADER,LAYOUT_FOOTER,LAYOUT_PAGE_NUMBERblocks — the document's visual structure, in multi-column-aware reading order.TABLES: structuredTABLE,CELL, andMERGED_CELLblocks — the actual cell grid, with merged-cell spans.FORMS:KEY_VALUE_SETpairs andSELECTION_ELEMENTblocks (checkboxes, radio buttons) with a selected/unselected status.
That is genuinely strong output — Textract's forms and tables extraction is a major reason AWS-native teams standardize on it. But the response is a block graph, not a document:
- No markdown, no prose. Nothing in the response is directly readable or embeddable; every consumer has to traverse blocks and relationships first.
- No cross-page hierarchy. A
LAYOUT_SECTION_HEADERon page 41 has no machine-readable link to the title block that opened the chapter on page 3. - No table-to-layout link. A
LAYOUT_TABLEregion and the structuredTABLEblock describing the same table share no Id relationship — matching them is left to you. - No figure content.
LAYOUT_FIGUREmarks where a figure sits, but Textract never returns cropped image bytes.
The gap between Blocks[] and good retrieval is where Textract-based RAG pipelines quietly fail.
Why the LAYOUT feature is required — and why flattening Blocks fails
Most Textract-to-RAG tutorials do the same thing: iterate LINE blocks, join with newlines, run a text splitter. This fails in two distinct ways.
Without LAYOUT, reading order is unrecoverable. LINE blocks carry geometry, not sequence. For single-column memos, sorting by vertical position happens to work. For the two-column reports, data sheets, and academic papers where Textract earns its keep, position-sorting interleaves the columns — line 1 of column A, line 1 of column B, line 2 of column A — into prose no human wrote and no chunker can repair. This is why POMA requires the LAYOUT feature for Textract input: a payload with only PAGE/LINE/WORD blocks fails transparently with a 422 telling you to re-run with FeatureTypes including "LAYOUT". Refusing to ingest is a feature — the alternative is scrambled multi-column text embedded silently into your index.
Even with LAYOUT, flattening discards the structure you paid for:
| What Textract computed | What Blocks-flattening + a character splitter does with it |
|---|---|
LAYOUT_* reading-order sequence | Ignored — lines joined by page position |
LAYOUT_TITLE / LAYOUT_SECTION_HEADER roles | Melted into undifferentiated prose |
Structured TABLE grids with MERGED_CELL spans | Cells read out row by row as loose words |
LAYOUT_HEADER / LAYOUT_FOOTER / LAYOUT_PAGE_NUMBER | Repeated on every page, embedded as noise |
SELECTION_ELEMENT checkbox state | Dropped — form answers vanish |
| Page numbers per block | Lost at concatenation — chunks can't cite a page |
Markdown conversion scripts that do honor the layout blocks are a step up, but they hand you display markdown — and then the same fragment problem every splitter has: the chunk under a page-41 subsection no longer knows which chapter it belongs to. For the full taxonomy, see the RAG chunking strategies guide.
The optimal chunker: reading the LAYOUT spine properly
POMA's bring-your-own-OCR (BYOCR) connector consumes the unmodified AnalyzeDocument response and runs only POMA's downstream stages on it:
- Shape validation, up front. The payload is fingerprinted (
Blocks[]with PascalCaseBlockTypeentries). A corrupted or mislabeled upload — or a Textract result missing LAYOUT blocks — fails immediately with a clear 422, never a silently degraded index. - Reading order from the layout spine. The
LAYOUT_*block sequence is followed as-is: it is Textract's own multi-column-aware reading order, so two-column pages come out as coherent prose, not interleaved lines. - Headings from layout roles.
LAYOUT_TITLEandLAYOUT_SECTION_HEADERblocks become markdown headings — the raw material for the hierarchy rebuild. - Tables spliced by geometry. Because Textract provides no Id link between a
LAYOUT_TABLEregion and its structuredTABLEblock, POMA matches them by bounding-box containment: aTABLEwhose box is ≥ 0.5 contained in aLAYOUT_TABLEregion replaces that region at its position in reading order. The cell grid is rendered as HTML, withMERGED_CELLblocks becomingrowspan/colspanattributes — embeddings never see half a table. StructuredTABLEblocks that no layout region claims are appended to the page, never dropped. - Figures counted, not lost.
LAYOUT_FIGUREhas a region but no bytes — Textract never returns crops — so each figure is counted as offloaded content incontent_metadata. The loss is visible in your ingest report, never silent. - Furniture stripped, form state kept. Header, footer, and page-number layout blocks are dropped as page furniture;
SELECTION_ELEMENTcheckboxes survive as selected/unselected marks (☒ / ☐) in the chunk text. - Hierarchy → chunks + chunksets. The cross-page heading tree is rebuilt — including POMA's heading-injection pass where the layout roles under-mark structure — and every sentence is emitted as a chunk with its
depth,page, andto_embednormalization, grouped into chunksets: root-to-leaf paths that keep each retrieved passage self-explanatory.
Because you already ran the OCR, POMA charges only for the downstream structure and chunking value — the OCR front-end is skipped entirely.
The result, on one reference legal document: the same query answered with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.
The AWS Textract RAG pipeline: chunk the output with POMA
Run Textract exactly as you do today — with LAYOUT in the feature list — save the raw response, and hand the JSON to PrimeCut:
import json
import os
import boto3
from poma import PrimeCut
# 1. Your existing Textract call — LAYOUT is required; add TABLES for cell grids.
textract = boto3.client("textract")
with open("contract.png", "rb") as f:
response = textract.analyze_document(
Document={"Bytes": f.read()},
FeatureTypes=["LAYOUT", "TABLES"],
)
# Multi-page PDFs: use start_document_analysis with the same FeatureTypes,
# collect the paged results, and save the combined Blocks the same way.
with open("contract.textract.json", "w") as f:
json.dump(response, f)
# 2. Chunk the raw result — POMA auto-detects the Textract shape.
client = PrimeCut() # reads POMA_API_KEY
result = client.ingest("contract.textract.json")
print(f"chunks: {len(result.chunks)}")
print(f"chunksets: {len(result.chunksets)}")
print(result.chunksets[0].to_embed) # embedding-ready, hierarchy includedTwo things worth knowing:
- Auto-detection. A JSON upload whose shape structurally fingerprints as a Textract result (
Blocks[]with PascalCaseBlockType) is routed through the BYOCR seam automatically. To force the declaration — or to opt a look-alike JSON out of detection — set theexternal_ocr_sourceparameter to"textract"or"none"respectively on the ingest request. - Dedicated endpoint. For explicit routing, the API exposes a
chunk_external_ocr_resultendpoint that acceptsexternal_ocr_source: "textract". See the API reference.
From result.chunks / result.chunksets onward, everything is your stack: your vector store, your embedding model, your retrieval strategy. See the Qdrant, LangChain, and LlamaIndex integrations.
Chunking options for Textract output, compared
| Approach | Reading order | Multi-column pages | Tables | Cross-page hierarchy | Figures & checkboxes |
|---|---|---|---|---|---|
| Flatten LINE blocks + character splitter | Position-sorted | Interleaved/scrambled | Cells melted into prose | ✗ | Dropped silently |
| Layout-to-markdown script + markdown splitter | LAYOUT sequence | ✓ | Depends on script | ✗ — isolated fragments | Usually dropped |
| Semantic chunker on flattened text | Position-sorted | Interleaved/scrambled | Fragile | ✗ | Dropped |
| POMA BYOCR (PrimeCut) | LAYOUT spine, as Textract computed it | ✓ | Spliced by geometry, HTML with row/colspans, unclaimed tables appended | ✓ chunksets | Figures counted visibly; ☒/☐ kept |
Frequently asked questions
How do I chunk AWS Textract output for RAG?
Don't flatten Blocks[] to plain text and run a character splitter — that discards the layout structure Textract computed and scrambles multi-column reading order. Call AnalyzeDocument with the LAYOUT feature (add TABLES for grids), save the raw JSON, and feed it to a structure-aware chunker. POMA's BYOCR connector accepts the unmodified response, follows the LAYOUT sequence, rebuilds the heading hierarchy, and emits chunks plus chunksets ready for embedding.
Why does POMA require the LAYOUT feature for Textract results?
Without LAYOUT, the response holds only PAGE/LINE/WORD blocks — geometry-sorted text with no reliable reading order. On multi-column documents, position-sorting interleaves the columns into scrambled prose no chunker can repair. POMA refuses to guess: a Textract payload without LAYOUT blocks fails transparently with a 422 telling you to re-run with FeatureTypes including "LAYOUT", rather than silently shipping a degraded index.
How do I convert Textract layout blocks to markdown for RAG?
Follow the LAYOUT_* sequence — it encodes multi-column-aware reading order — and map roles to markdown: LAYOUT_TITLE and LAYOUT_SECTION_HEADER become headings, LAYOUT_TEXT body prose, LAYOUT_LIST list items; header, footer, and page-number blocks are furniture to drop. POMA does this mapping on the raw response, then rebuilds the cross-page hierarchy into chunks and chunksets.
How are Textract tables handled during chunking?
Textract provides no Id link between a LAYOUT_TABLE region and its structured TABLE block, so POMA splices them by bounding-box geometry (≥ 0.5 containment) and renders the cell grid as HTML, with MERGED_CELL blocks becoming rowspan/colspan attributes. TABLE blocks no layout region claims are appended, never dropped.
What happens to figures and checkboxes in a Textract result?
Textract never returns cropped image bytes, so each LAYOUT_FIGURE is counted as offloaded content in content_metadata — the loss is visible in your ingest report, never silent. SELECTION_ELEMENT checkboxes survive as ☒ / ☐ marks in the chunk text, so form state reaches retrieval.
Should I switch from Textract to another OCR to fix retrieval quality?
Usually not. If Textract fits your stack — AWS-native infrastructure, strong forms and tables, documents that never leave your account — keep it. Retrieval quality is mostly decided by what happens to the Blocks output next, and a BYOCR chunker closes that gap without touching your OCR contract.
From Textract to your vector database
End-to-end recipes for wiring Textract output through POMA into your store:
- AWS Textract → Chroma
- AWS Textract → Elasticsearch
- AWS Textract → LanceDB
- AWS Textract → Milvus
- AWS Textract → MongoDB Atlas
- AWS Textract → OpenSearch
- AWS Textract → pgvector
- AWS Textract → Pinecone
- AWS Textract → Qdrant
- AWS Textract → Redis
- AWS Textract → Turbopuffer
- AWS Textract → Vespa
- AWS Textract → Weaviate
The rest of the series
The optimal chunker, for every OCR and parsing tool you already run:
- Azure Document Intelligence
- DeepSeek-OCR
- Docling
- LlamaParse
- Marker
- MinerU
- Mistral OCR
- PaddleOCR-VL
- Unstructured.io
Or start with the fundamentals: RAG chunking guide · Document ingestion guide · POMA chunksets.