Skip to content

AWS Textract to Weaviate: Chunking for RAG ​

The short answer: AWS Textract gives you an excellent block graph — layout regions in multi-column reading order, structured tables, isolated page furniture; Weaviate gives you excellent hybrid retrieval — BM25F and vector search fused in one query. Wired together naively — flatten Blocks[] to LINE text, split, insert — the pipeline poisons both signals at once: scrambled reading order degrades the vectors, and repeated headers and footers inflate the keyword scores. The missing link is POMA: PrimeCut().ingest() consumes the raw AnalyzeDocument response (auto-detected), reads the LAYOUT spine, drops the furniture Textract already isolated, and emits hierarchy-preserving chunksets whose fields map directly onto Weaviate's filterable properties.

What each end of the pipeline actually provides ​

AWS Textract (AnalyzeDocument with FeatureTypes=["LAYOUT", "TABLES"]) returns a flat Blocks[] array: LAYOUT_* blocks in multi-column-aware reading order, LAYOUT_TITLE and LAYOUT_SECTION_HEADER roles, structured TABLE/CELL/MERGED_CELL grids, checkbox state — and, crucially for this pairing, page furniture already isolated as LAYOUT_HEADER, LAYOUT_FOOTER, and LAYOUT_PAGE_NUMBER blocks. What it doesn't provide: markdown, cross-page hierarchy, or retrieval units. Details: The Optimal Chunker for AWS Textract.

Weaviate provides collections with named vectors, built-in hybrid search (BM25F keyword scoring fused with vector similarity via the alpha blend), filterable properties, cross-references between collections, and generative/reranker modules. What it doesn't provide: any opinion about what an object should contain. BM25F scores whatever text you indexed — including a confidentiality banner repeated on all 80 pages, if that's what you gave it. Details: The Optimal Chunks for the Best Retrieval in Weaviate.

The naive wiring, and where it breaks ​

The common recipe — flatten LINE blocks to text, split, insert into a collection, query hybrid — breaks in a pair-specific way, because Weaviate's hybrid search has two signals to poison:

  • The keyword side drowns in page furniture. Position-sorted LINE flattening drags every page's running header, footer, and page number into the chunks. BM25F then does its job faithfully: tokens from the document title and the "CONFIDENTIAL" banner appear in nearly every object, so half of hybrid scoring is spent rewarding text that answers nothing. The cruel part: Textract had already segregated this furniture into LAYOUT_HEADER/LAYOUT_FOOTER/LAYOUT_PAGE_NUMBER blocks — the flattening step mixed it back in.
  • The vector side embeds scrambled prose. LINE blocks carry geometry, not sequence; position-sorting interleaves multi-column pages into text no human wrote, and the embedding encodes the garble without complaint.
  • Hierarchy and pages are melted away, so retrieved objects arrive without lineage, no property can cite a page, and alpha tuning becomes an exercise in polishing noise.

POMA fixes both signals in one pass: it follows the LAYOUT_* reading order, drops the furniture blocks, promotes layout roles to headings, rebuilds the cross-page tree, and emits overlap-free chunksets whose text is clean for BM25F and coherent for the embedding. A Textract payload without LAYOUT blocks fails transparently with a 422 telling you to re-run with FeatureTypes including "LAYOUT" — never scrambled text in the collection.

The pipeline, end to end ​

bash
pip install boto3 weaviate-client sentence-transformers poma
python
import json
import os

import boto3
import weaviate
from poma import PrimeCut
from sentence_transformers import SentenceTransformer
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.init import Auth
from weaviate.classes.query import Filter

# 1. Textract — your existing call; LAYOUT is required, TABLES recommended.
textract = boto3.client("textract")
with open("contract.png", "rb") as f:
    response = textract.analyze_document(
        Document={"Bytes": f.read()},
        FeatureTypes=["LAYOUT", "TABLES"],
    )
with open("contract.textract.json", "w") as f:
    json.dump(response, f)

# 2. The missing link — raw Blocks[] in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("contract.textract.json")  # Textract shape auto-detected

# 3. Weaviate — one chunkset collection, self-provided vectors.
client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)
chunksets = client.collections.create(
    "ContractChunkset",
    vectorizer_config=Configure.Vectorizer.none(),
    properties=[
        Property(name="content", data_type=DataType.TEXT),
        Property(name="file_id", data_type=DataType.TEXT),
        Property(name="page", data_type=DataType.INT),
        Property(name="depth", data_type=DataType.INT),
        Property(name="chunk_index", data_type=DataType.INT),
    ],
)

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
with chunksets.batch.dynamic() as batch:
    for chunkset in result.chunksets:
        leaf = chunkset.chunks[-1]  # deepest chunk in the root-to-leaf path
        batch.add_object(
            properties={
                "content": chunkset.to_embed,
                "file_id": leaf.file_id,
                "page": leaf.page,
                "depth": leaf.depth,
                "chunk_index": leaf.chunk_index,
            },
            vector=model.encode(chunkset.to_embed).tolist(),
        )

# 4. Hybrid query — BM25F + vector, fused by alpha, filtered by document.
question = "Which termination clauses require written notice?"
hits = chunksets.query.hybrid(
    query=question,
    vector=model.encode(question).tolist(),
    alpha=0.5,  # 0 = pure BM25F, 1 = pure vector; tune per corpus
    limit=5,
    filters=Filter.by_property("file_id").equal(leaf.file_id),
)
for obj in hits.objects:
    print(obj.properties["page"], obj.properties["content"][:80])
client.close()

Upstream of step 3, POMA validates the payload shape (a corrupted or mislabeled upload 422s immediately), splices structured TABLE blocks into LAYOUT_TABLE regions by bounding-box geometry (≥ 0.5 containment — Textract provides no Id link), renders grids as HTML with rowspan/colspan from MERGED_CELL blocks, keeps SELECTION_ELEMENT checkboxes as ☒ / ☐ marks, and counts byte-less LAYOUT_FIGURE regions as offloaded content in content_metadata — visible loss, never silent. To force or suppress detection, set external_ocr_source to "textract" or "none". If you want navigable lineage on top, add a Chunk collection and cross-reference each chunkset to its member chunks — optional, since every chunkset already carries its full root-to-leaf context in the text itself.

Measured on our reference legal-document benchmark, this pipeline answers the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss. Methodology: Document Ingestion & Chunking for RAG.

Metadata mapping: POMA fields → Weaviate primitives ​

POMA chunk fieldWeaviate primitiveWhat it enables
to_embedcontent TEXT property + self-provided vectorBM25F keyword side and vector side of hybrid, from the same clean text
file_idTEXT property, Filter.by_propertyscope queries to one document
pageINT property, Filter.by_propertypage-cited answers, page-range filters
depthINT property, Filter.by_propertyhierarchy-aware filtering and re-ranking
chunk_indexINT propertystable ordering at assembly time
chunkset ↔ chunk lineagecross-references between collectionsnavigate from a retrieved chunkset to per-sentence citations

Frequently asked questions ​

How do I get AWS Textract results into Weaviate for RAG? ​

Call AnalyzeDocument with FeatureTypes including LAYOUT (add TABLES for grids), save the raw JSON, and run PrimeCut().ingest() on it — auto-detected, LAYOUT reading order followed, hierarchy rebuilt. Create a chunkset collection with file_id/page/depth properties, insert each chunkset with its vector, and query with Weaviate's built-in hybrid search.

Why does my Textract-to-Weaviate hybrid search keep matching page headers and footers? ​

Because LINE flattening repeats every page's running header, footer, and page number inside the chunks, and BM25F rewards that repetition — furniture tokens appear in nearly every object. Textract already isolates this furniture as LAYOUT_HEADER/LAYOUT_FOOTER/LAYOUT_PAGE_NUMBER blocks; POMA drops those regions before chunking, so the keyword side scores only real content.

What alpha should I use for hybrid queries over Textract content? ​

alpha=0 is pure BM25F, alpha=1 pure vector; 0.5 is a sensible start for OCR'd business documents. Lower it for exact-token-heavy queries (clause numbers, field labels), raise it for paraphrase questions — and remember the tuning is only meaningful once the indexed text is furniture-free.

Which Weaviate properties should Textract chunks carry? ​

content (TEXT — what BM25F scores), file_id (TEXT), and page/depth/chunk_index (INT), all filterable with Filter.by_property. Provide the vector at insert time or configure a vectorizer on the collection.

Can Weaviate cross-references model POMA's chunk-to-chunkset hierarchy? ​

Yes — keep a Chunk and a Chunkset collection with references from each chunkset to its member chunks for per-sentence citations. It's optional for quality: every chunkset already carries its full root-to-leaf lineage in its own text.

Same parser, different store: Textract → Qdrant · Textract → Pinecone · Textract → Chroma

Same store, different parser: Mistral OCR → Weaviate · Docling → Weaviate · Unstructured → Weaviate

Foundations: The Optimal Chunker for AWS Textract · The Optimal Chunks for Weaviate · All pipeline recipes · RAG chunking guide