Skip to content

AWS Textract to Milvus: Chunking for RAG ​

The short answer: AWS Textract gives you a LAYOUT-annotated block graph; Milvus gives you billion-scale ANN search with scalar-field filtering and partitions. Wired together naively — flatten Blocks[], split, embed, insert — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy. The missing link is POMA: PrimeCut().ingest() consumes the raw AnalyzeDocument JSON (auto-detected), emits hierarchy-preserving chunksets, and those map cleanly onto a Milvus schema — hierarchy metadata as scalar fields, file_id as partition key, to_embed as the vector source — with cheatsheet assembly after retrieval.

What each end of the pipeline actually provides ​

AWS Textract (AnalyzeDocument with FeatureTypes: ["LAYOUT", "TABLES"]) returns a flat Blocks[] graph: LAYOUT_* blocks in multi-column-aware reading order, LAYOUT_TITLE and LAYOUT_SECTION_HEADER marking structure, structured TABLE grids with MERGED_CELL spans. What it doesn't provide: markdown, cross-page hierarchy, or retrieval units — and without the LAYOUT feature, not even reliable reading order. Details: The Optimal Chunker for AWS Textract.

Milvus provides collections with declared schemas, partitions with partition-key routing, scalar fields usable in filter expressions during ANN search, a choice of HNSW, IVF, and DiskANN index types, and — in recent versions — sparse vectors with built-in BM25 for hybrid search, self-hosted or on Zilliz Cloud. What it doesn't provide: any opinion about what an entity should contain. It retrieves nearest neighbors of whatever you embedded — including scrambled Block fragments, if that's what you inserted. Details: The Optimal Chunks for the Best Retrieval in Milvus.

The naive wiring, and where it breaks ​

The common recipe — iterate LINE blocks, join with newlines, RecursiveCharacterTextSplitter, embed, insert into a (id, vector, text) collection — breaks in a pair-specific way:

  • The schema freezes before the metadata exists. Milvus schemas are fixed at collection creation, and the naive pipeline creates (id, vector, text) on day one. Because flattening destroyed Textract's page numbers and heading roles before the first insert, page, depth, and file_id never become scalar fields — and retrofitting them into a live collection means re-ingesting everything. Filtered search and partition pruning are off the table for good.
  • Textract's reading order dies at the flatten. LINE blocks carry geometry, not sequence; position-sorting interleaves the columns of the two-column reports Textract is bought for, and Milvus faithfully indexes the scrambled prose at any scale you pay for.
  • Overlap inflates every index type. Splitter overlap embeds boundary spans twice: a bigger HNSW graph, fatter IVF lists, more DiskANN pages — and top-k results where hits 2 and 3 are near-duplicates of hit 1.

The pipeline, end to end ​

bash
pip install boto3 poma pymilvus sentence-transformers
python
import json
import os

import boto3
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from pymilvus import DataType, MilvusClient
from sentence_transformers import SentenceTransformer

# 1. Textract — your existing call, unchanged. LAYOUT is required.
textract = boto3.client("textract")
with open("contract.png", "rb") as f:
    response = textract.analyze_document(
        Document={"Bytes": f.read()},
        FeatureTypes=["LAYOUT", "TABLES"],
    )
with open("contract.textract.json", "w") as f:
    json.dump(response, f)

# 2. The missing link — raw Blocks[] in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.textract.json",  # Textract shape auto-detected
    download_dir="store",
    filename="contract.poma",  # keep the archive — page numbers live in its chunk records
)

# A chunkset's structural metadata comes from its leaf chunk (cs.chunks is in
# document order, so the last index is the leaf). depth is on the SDK objects;
# page only exists on the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
    c["chunk_index"]: c.get("page")
    for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}

# 3. Milvus — declare hierarchy as scalar fields BEFORE the first insert.
client = MilvusClient(uri=os.environ["MILVUS_URI"], token=os.environ["MILVUS_TOKEN"])

schema = MilvusClient.create_schema(auto_id=True)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=384)
schema.add_field("file_id", DataType.VARCHAR, max_length=512, is_partition_key=True)
schema.add_field("page", DataType.INT64)
schema.add_field("depth", DataType.INT64)
schema.add_field("chunkset_index", DataType.INT64)
schema.add_field("to_embed", DataType.VARCHAR, max_length=65535)

index_params = client.prepare_index_params()
index_params.add_index(field_name="embedding", index_type="HNSW", metric_type="COSINE")

client.create_collection("contracts", schema=schema, index_params=index_params)

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
embeddings = model.encode([cs.to_embed for cs in result.chunksets])

client.insert(
    collection_name="contracts",
    data=[
        {
            "embedding": emb.tolist(),
            "file_id": cs.file_id,
            "chunkset_index": cs.chunkset_index,
            "page": page_of[cs.chunks[-1]],
            "depth": depth_of[cs.chunks[-1]],
            "to_embed": cs.to_embed,
        }
        for cs, emb in zip(result.chunksets, embeddings)
    ],
)

# 4. Filtered ANN search — the partition key prunes to one document's data.
hits = client.search(
    collection_name="contracts",
    data=model.encode(["What are the early termination conditions?"]).tolist(),
    limit=3,
    filter=f'file_id == "{result.chunksets[0].file_id}"',
    output_fields=["to_embed", "file_id", "page", "depth", "chunkset_index"],
)

POMA validates the payload up front: a Textract result missing LAYOUT blocks fails with a clear 422 telling you to re-run with FeatureTypes including "LAYOUT" — never scrambled multi-column prose embedded into your collection. Tables are spliced by geometry and arrive as HTML with rowspan/colspan; figures (Textract returns no crops) are counted as offloaded content in content_metadata. To force or suppress detection, set external_ocr_source to "textract" or "none".

Because every retrieved chunkset is self-explanatory — leaf plus all ancestor breadcrumbs — the hits can be deduplicated and merged client-side into one prompt-ready cheatsheet. On our reference legal document, that assembly answers the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss. Methodology: Document Ingestion & Chunking for RAG.

Partitions, filters, and index choice ​

Three Milvus capabilities do real work in this pipeline, and all three depend on the metadata surviving ingestion:

  • Partition key on file_id routes each document's chunksets together, so document-scoped queries — the dominant pattern for Textract corpora like claims and contracts — touch only the relevant partitions. Multi-tenant deployments use a tenant identifier instead and filter file_id as a plain scalar.
  • Scalar filters during ANN search (page <= 40, depth == 2) enable page-cited answers and hierarchy-aware retrieval. They only work because POMA carried page and depth through from Textract's block graph — depth on the SDK's chunk objects, page on the chunk records inside the .poma archive.
  • Hybrid sparse + dense. Recent Milvus versions add sparse vectors with built-in BM25. Textract-processed documents are dense with exact tokens — form field values, clause numbers, IDs — that embeddings blur; hybrid search catches both these and paraphrase queries.

Index choice (HNSW for in-memory recall, IVF for memory economy, DiskANN for larger-than-RAM corpora) is orthogonal — but overlap-free chunksets shrink whichever index you pick, because no boundary span is ever embedded twice.

Metadata mapping: POMA fields → Milvus primitives ​

POMA chunk fieldMilvus primitiveWhat it enables
to_embedFLOAT_VECTOR field + HNSW/IVF/DiskANN index; optional sparse/BM25 fieldparaphrase + exact-term hybrid retrieval
file_idVARCHAR scalar field, is_partition_key=Truepartition pruning, per-document scoping
page (on the archive's chunk records; take the chunkset's leaf chunk)INT64 scalar field, filter expressionpage-cited answers, page-range filters
depth of the leaf chunkINT64 scalar fieldfilter or re-rank by hierarchy level
chunkset_indexINT64 scalar fieldstable ordering at assembly time
chunkset lineageroot-to-leaf text carried in the to_embed VARCHAR fieldself-explanatory hits, client-side cheatsheet assembly

Frequently asked questions ​

How do I get AWS Textract results into Milvus for RAG? ​

Save the raw AnalyzeDocument JSON (with LAYOUT in the feature list), run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), create a collection whose schema declares file_id/chunkset_index/page/depth as scalar fields, embed each chunkset's to_embed, and insert. Retrieve with filtered ANN search plus cheatsheet assembly.

What scalar fields should a Milvus collection define for Textract chunks? ​

file_id as VARCHAR (ideally the partition key), chunkset_index, page, and depth as INT64, plus a VARCHAR for the chunkset text next to the FLOAT_VECTOR field. chunkset_index and depth come off the SDK objects; page comes from the .poma archive's chunk records, read from the chunkset's leaf chunk. Schemas are fixed at creation — declare them before the first insert.

Should file_id be a Milvus partition key for Textract documents? ​

For multi-document corpora, yes: partition-key routing means document-scoped searches touch only the relevant partitions. Tenant-scoped deployments use a tenant identifier as the key and filter file_id as a plain scalar.

Which Milvus index type should I use for Textract chunkset embeddings? ​

HNSW for in-memory recall, IVF for memory economy, DiskANN for larger-than-RAM corpora. All of them get smaller and cleaner with overlap-free chunksets — no near-duplicate vectors, no wasted top-k slots.

Does Milvus hybrid sparse and dense search help with Textract output? ​

Yes. Recent Milvus versions support sparse vectors with built-in BM25 next to dense embeddings. Textract documents are full of exact tokens (form values, clause numbers, IDs) that hybrid search catches and dense-only search blurs.

Same parser, different store: Textract → Qdrant · Textract → pgvector · Textract → Chroma

Same store, different parser: Mistral OCR → Milvus · Unstructured.io → Milvus · Marker → Milvus

Foundations: The Optimal Chunker for AWS Textract · The Optimal Chunks for Milvus · All pipeline recipes · RAG chunking guide