AWS Textract to Weaviate: Chunking for RAG
The short answer: AWS Textract gives you an excellent block graph — layout regions in multi-column reading order, structured tables, isolated page furniture; Weaviate gives you excellent hybrid retrieval — BM25F and vector search fused in one query. Wired together naively — flatten Blocks[] to LINE text, split, insert — the pipeline poisons both signals at once: scrambled reading order degrades the vectors, and repeated headers and footers inflate the keyword scores. The missing link is POMA: PrimeCut().ingest() consumes the raw AnalyzeDocument response (auto-detected), reads the LAYOUT spine, drops the furniture Textract already isolated, and emits hierarchy-preserving chunksets whose fields map directly onto Weaviate's filterable properties.
What each end of the pipeline actually provides
AWS Textract (AnalyzeDocument with FeatureTypes=["LAYOUT", "TABLES"]) returns a flat Blocks[] array: LAYOUT_* blocks in multi-column-aware reading order, LAYOUT_TITLE and LAYOUT_SECTION_HEADER roles, structured TABLE/CELL/MERGED_CELL grids, checkbox state — and, crucially for this pairing, page furniture already isolated as LAYOUT_HEADER, LAYOUT_FOOTER, and LAYOUT_PAGE_NUMBER blocks. What it doesn't provide: markdown, cross-page hierarchy, or retrieval units. Details: The Optimal Chunker for AWS Textract.
Weaviate provides collections with named vectors, built-in hybrid search (BM25F keyword scoring fused with vector similarity via the alpha blend), filterable properties, cross-references between collections, and generative/reranker modules. What it doesn't provide: any opinion about what an object should contain. BM25F scores whatever text you indexed — including a confidentiality banner repeated on all 80 pages, if that's what you gave it. Details: The Optimal Chunks for the Best Retrieval in Weaviate.
The naive wiring, and where it breaks
The common recipe — flatten LINE blocks to text, split, insert into a collection, query hybrid — breaks in a pair-specific way, because Weaviate's hybrid search has two signals to poison:
- The keyword side drowns in page furniture. Position-sorted LINE flattening drags every page's running header, footer, and page number into the chunks. BM25F then does its job faithfully: tokens from the document title and the "CONFIDENTIAL" banner appear in nearly every object, so half of hybrid scoring is spent rewarding text that answers nothing. The cruel part: Textract had already segregated this furniture into
LAYOUT_HEADER/LAYOUT_FOOTER/LAYOUT_PAGE_NUMBERblocks — the flattening step mixed it back in. - The vector side embeds scrambled prose. LINE blocks carry geometry, not sequence; position-sorting interleaves multi-column pages into text no human wrote, and the embedding encodes the garble without complaint.
- Hierarchy and pages are melted away, so retrieved objects arrive without lineage, no property can cite a page, and
alphatuning becomes an exercise in polishing noise.
POMA fixes both signals in one pass: it follows the LAYOUT_* reading order, drops the furniture blocks, promotes layout roles to headings, rebuilds the cross-page tree, and emits overlap-free chunksets whose text is clean for BM25F and coherent for the embedding. A Textract payload without LAYOUT blocks fails transparently with a 422 telling you to re-run with FeatureTypes including "LAYOUT" — never scrambled text in the collection.
The pipeline, end to end
pip install boto3 weaviate-client sentence-transformers pomaimport json
import os
import boto3
import weaviate
from poma import PrimeCut
from sentence_transformers import SentenceTransformer
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.init import Auth
from weaviate.classes.query import Filter
# 1. Textract — your existing call; LAYOUT is required, TABLES recommended.
textract = boto3.client("textract")
with open("contract.png", "rb") as f:
response = textract.analyze_document(
Document={"Bytes": f.read()},
FeatureTypes=["LAYOUT", "TABLES"],
)
with open("contract.textract.json", "w") as f:
json.dump(response, f)
# 2. The missing link — raw Blocks[] in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest("contract.textract.json") # Textract shape auto-detected
# 3. Weaviate — one chunkset collection, self-provided vectors.
client = weaviate.connect_to_weaviate_cloud(
cluster_url=os.environ["WEAVIATE_URL"],
auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)
chunksets = client.collections.create(
"ContractChunkset",
vectorizer_config=Configure.Vectorizer.none(),
properties=[
Property(name="content", data_type=DataType.TEXT),
Property(name="file_id", data_type=DataType.TEXT),
Property(name="page", data_type=DataType.INT),
Property(name="depth", data_type=DataType.INT),
Property(name="chunk_index", data_type=DataType.INT),
],
)
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
with chunksets.batch.dynamic() as batch:
for chunkset in result.chunksets:
leaf = chunkset.chunks[-1] # deepest chunk in the root-to-leaf path
batch.add_object(
properties={
"content": chunkset.to_embed,
"file_id": leaf.file_id,
"page": leaf.page,
"depth": leaf.depth,
"chunk_index": leaf.chunk_index,
},
vector=model.encode(chunkset.to_embed).tolist(),
)
# 4. Hybrid query — BM25F + vector, fused by alpha, filtered by document.
question = "Which termination clauses require written notice?"
hits = chunksets.query.hybrid(
query=question,
vector=model.encode(question).tolist(),
alpha=0.5, # 0 = pure BM25F, 1 = pure vector; tune per corpus
limit=5,
filters=Filter.by_property("file_id").equal(leaf.file_id),
)
for obj in hits.objects:
print(obj.properties["page"], obj.properties["content"][:80])
client.close()Upstream of step 3, POMA validates the payload shape (a corrupted or mislabeled upload 422s immediately), splices structured TABLE blocks into LAYOUT_TABLE regions by bounding-box geometry (≥ 0.5 containment — Textract provides no Id link), renders grids as HTML with rowspan/colspan from MERGED_CELL blocks, keeps SELECTION_ELEMENT checkboxes as ☒ / ☐ marks, and counts byte-less LAYOUT_FIGURE regions as offloaded content in content_metadata — visible loss, never silent. To force or suppress detection, set external_ocr_source to "textract" or "none". If you want navigable lineage on top, add a Chunk collection and cross-reference each chunkset to its member chunks — optional, since every chunkset already carries its full root-to-leaf context in the text itself.
Measured on our reference legal-document benchmark, this pipeline answers the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss. Methodology: Document Ingestion & Chunking for RAG.
Metadata mapping: POMA fields → Weaviate primitives
| POMA chunk field | Weaviate primitive | What it enables |
|---|---|---|
to_embed | content TEXT property + self-provided vector | BM25F keyword side and vector side of hybrid, from the same clean text |
file_id | TEXT property, Filter.by_property | scope queries to one document |
page | INT property, Filter.by_property | page-cited answers, page-range filters |
depth | INT property, Filter.by_property | hierarchy-aware filtering and re-ranking |
chunk_index | INT property | stable ordering at assembly time |
| chunkset ↔ chunk lineage | cross-references between collections | navigate from a retrieved chunkset to per-sentence citations |
Frequently asked questions
How do I get AWS Textract results into Weaviate for RAG?
Call AnalyzeDocument with FeatureTypes including LAYOUT (add TABLES for grids), save the raw JSON, and run PrimeCut().ingest() on it — auto-detected, LAYOUT reading order followed, hierarchy rebuilt. Create a chunkset collection with file_id/page/depth properties, insert each chunkset with its vector, and query with Weaviate's built-in hybrid search.
Why does my Textract-to-Weaviate hybrid search keep matching page headers and footers?
Because LINE flattening repeats every page's running header, footer, and page number inside the chunks, and BM25F rewards that repetition — furniture tokens appear in nearly every object. Textract already isolates this furniture as LAYOUT_HEADER/LAYOUT_FOOTER/LAYOUT_PAGE_NUMBER blocks; POMA drops those regions before chunking, so the keyword side scores only real content.
What alpha should I use for hybrid queries over Textract content?
alpha=0 is pure BM25F, alpha=1 pure vector; 0.5 is a sensible start for OCR'd business documents. Lower it for exact-token-heavy queries (clause numbers, field labels), raise it for paraphrase questions — and remember the tuning is only meaningful once the indexed text is furniture-free.
Which Weaviate properties should Textract chunks carry?
content (TEXT — what BM25F scores), file_id (TEXT), and page/depth/chunk_index (INT), all filterable with Filter.by_property. Provide the vector at insert time or configure a vectorizer on the collection.
Can Weaviate cross-references model POMA's chunk-to-chunkset hierarchy?
Yes — keep a Chunk and a Chunkset collection with references from each chunkset to its member chunks for per-sentence citations. It's optional for quality: every chunkset already carries its full root-to-leaf lineage in its own text.
Related recipes
Same parser, different store: Textract → Qdrant · Textract → Pinecone · Textract → Chroma
Same store, different parser: Mistral OCR → Weaviate · Docling → Weaviate · Unstructured → Weaviate
Foundations: The Optimal Chunker for AWS Textract · The Optimal Chunks for Weaviate · All pipeline recipes · RAG chunking guide