AWS Textract to Vespa: Chunking for RAG
The short answer: Textract's AnalyzeDocument with LAYOUT gives you multi-column-aware reading order and structured tables; Vespa gives you one system that natively fuses BM25 and ANN over tensor fields declared in a schema. Wired together naively — flatten Blocks[], split, embed — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy or fits Vespa's schema-first model. The missing link is POMA: PrimeCut().ingest() consumes the raw Textract response, emits hierarchy-preserving chunksets into a .poma archive, and poma.vektoria feeds content-free documents to Vespa — file_id, chunkset_index, and the embedding tensor — while the actual text lives on a volume. Retrieval runs your normal Vespa query, then assemble() fetches matching content and returns prompt-ready cheatsheets.
What each end of the pipeline actually provides
AWS Textract (AnalyzeDocument) returns a flat Blocks[] graph. Without LAYOUT, only PAGE/LINE/WORD blocks exist and reading order isn't reliably reconstructible; with LAYOUT, the LAYOUT_* sequence is Textract's own multi-column-aware reading order, LAYOUT_TITLE/LAYOUT_SECTION_HEADER mark headings, and TABLES adds structured TABLE/MERGED_CELL blocks spliced to LAYOUT_TABLE regions by geometry (no Id link exists between them). What it doesn't provide: markdown, cross-page hierarchy, or retrieval units. Details: The Optimal Chunker for AWS Textract.
Vespa stores vectors as ordinary tensor fields inside the same document type as scalar attributes — no separate vector-store object — and fuses BM25 and ANN in one YQL query via a rank profile. Its document type is compiled into an application package at deploy time; there's no dynamic schema. What it doesn't provide: any opinion about what a document should represent, or a way to filter on a field you never declared. Details: Vespa Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — flatten Blocks[] (or the LAYOUT_* sequence) into one string per page, embed it into a bare text/embedding schema, deploy, iterate — breaks in a pair-specific way. Because Vespa's fields are fixed at deploy time, a schema built around "just the text and the vector" has no file_id or chunkset_index attribute when the team later needs per-document filtering or wants assemble()-compatible retrieval — that requires a schema change and a redeploy, not a query tweak. Compounding it, Textract's table splice (LAYOUT_TABLE matched to TABLE by bounding-box geometry) never happened upstream, so the flattened text also carries melted-together table cells into the one field Vespa does have. Two separate fixes, both avoidable by declaring the right fields before the first row is written.
The pipeline, end to end
pip install boto3 pyvespaimport json
import boto3
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from vespa.package import ApplicationPackage, Field, Schema, Document, HNSW, RankProfile
from vespa.deployment import VespaDocker
# 1. Your existing Textract call — LAYOUT is required; add TABLES for cell grids.
textract = boto3.client("textract")
with open("contract.png", "rb") as f:
response = textract.analyze_document(
Document={"Bytes": f.read()},
FeatureTypes=["LAYOUT", "TABLES"],
)
with open("contract.textract.json", "w") as f:
json.dump(response, f)
# 2. The missing link — raw result JSON in, .poma archive out (the volume's source of truth).
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.textract.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
package = ApplicationPackage(
name="contracts",
schema=[Schema(
name="poma",
document=Document(fields=[
Field(name="file_id", type="string", indexing=["attribute"]),
Field(name="chunkset_index", type="int", indexing=["attribute"]),
Field(
name="embedding", type="tensor<float>(x[384])",
indexing=["attribute", "index"],
ann=HNSW(distance_metric="angular"),
),
]),
rank_profiles=[RankProfile(
name="poma_rank",
inputs=[("query(q)", "tensor<float>(x[384])")],
first_phase="closeness(field, embedding)",
)],
)],
)
sess = VespaDocker().deploy(application_package=package)
# 3. Content-free ingest — Vespa holds only routing attributes; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
docs = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
docs.append({"id": r.id, "fields": {
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
"embedding": embedder.embed([r.text])[0],
}})
sess.feed_iterable(docs, schema="poma", namespace="contracts")
# 4. Retrieval — Vespa's verified query shape (fields selected explicitly), then assemble.
qv = embedder.embed(["early termination conditions"])[0]
resp = sess.query(yql="select file_id, chunkset_index from poma where "
"{targetHits:100}nearestNeighbor(embedding, q) limit 10",
ranking="poma_rank", body={"input.query(q)": qv})
context = assemble(resp, volume=vol) # -> [{"file_id", "content"}, ...]POMA validates the Textract payload up front — a response missing LAYOUT blocks 422s immediately rather than shipping scrambled reading order — and rebuilds the cross-page heading tree before chunking. Retrieved chunksets share ancestor lineage across a document; assemble() deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answers our reference legal-document benchmark with 337 tokens instead of 1,542 for a recursive-splitter baseline. Methodology here.
Metadata mapping: POMA fields → Vespa primitives
POMA field (vektoria Record) | Vespa primitive | What it enables |
|---|---|---|
r.text (chunkset to_embed) | tensor<float>(x[N]) field, HNSW-indexed | ANN via nearestNeighbor(embedding, q) |
r.id (chunkset_uuid) | document id | deterministic identity — re-feed overwrites, never duplicates |
payload["file_id"] | string attribute field | scope YQL predicates to one document; must be select-ed for assemble() |
payload["chunkset_index"] | int attribute field | maps a hit back to volume content; must be select-ed for assemble() |
payload["chunks"] / text | written to the volume, not the document | full chunkset content (incl. spliced Textract tables) reconstructed at retrieval, never a Vespa field |
Frequently asked questions
How do I get AWS Textract output into Vespa for RAG?
Call AnalyzeDocument with LAYOUT (and TABLES) as usual, save the JSON, and run it through PrimeCut.ingest to get a .poma archive. Read it with records_from_archive, write each chunkset's content to a volume, and feed content-free documents — file_id, chunkset_index, and the tensor embedding — into a Vespa document type. Query with nearestNeighbor, select file_id and chunkset_index explicitly, then hand the response to assemble().
Why does flattening Textract's Blocks[] break Vespa's schema-first design?
Vespa's document type is declared once, in the application package, before you deploy — there's no ad hoc field added at query time. A pipeline that flattens Blocks[] to one string per page and only later realizes it needs file_id or chunkset_index for filtering has to change the schema and redeploy the application, not just adjust a query. Planning the fields up front avoids that detour entirely.
What fields does a Vespa document type need for Textract chunksets?
Two attribute fields — file_id (string) and chunkset_index (int) — plus a tensor field carrying the embedding with its own HNSW index, all inside one document type. Chunk and chunkset content is not a field at all; it's written to a volume, keeping the schema stable regardless of how large a chunkset's text or spliced table HTML gets.
Why does assemble() return nothing even though my Vespa query has hits?
Vespa doesn't return document fields unless the YQL selects them. A query like select * from sources * where nearestNeighbor(...) without file_id and chunkset_index in the select clause gets hits with no usable metadata, and assemble() has nothing to look up on the volume. Selecting those two fields explicitly in every retrieval query is the fix.
Are Textract's tables and figures preserved through to Vespa retrieval?
Yes, at the ingest layer — PrimeCut splices Textract's structured TABLE blocks into their LAYOUT_TABLE region by geometry and renders the grid as HTML before it ever reaches Vespa, and counts figures Textract can't return bytes for as offloaded content. None of that lives in the Vespa document itself; it's part of the chunkset content assemble() fetches from the volume after retrieval.
Related recipes
Same parser, different store: Textract → Turbopuffer · Textract → Elasticsearch · Textract → OpenSearch
Also available: Textract → pgvector · Textract → Milvus · Textract → Chroma
Foundations: The Optimal Chunker for AWS Textract · Vespa Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide