Skip to content

AWS Textract to pgvector: Chunking for RAG ​

The short answer: AWS Textract gives you a LAYOUT-annotated block graph; Postgres with pgvector gives you ANN search inside the relational database you already run. Wired together naively — flatten Blocks[], split, embed, insert — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy. The missing link is POMA: PrimeCut().ingest() consumes the raw AnalyzeDocument JSON (auto-detected), emits hierarchy-preserving chunksets, and those land in Postgres as two joined tables — so a single SQL statement runs the ANN search and the lineage JOIN, over data that was ingested transactionally and is hybrid-searchable via tsvector.

What each end of the pipeline actually provides ​

AWS Textract (AnalyzeDocument with FeatureTypes: ["LAYOUT", "TABLES"]) returns a flat Blocks[] graph: LAYOUT_* blocks in multi-column-aware reading order, LAYOUT_TITLE and LAYOUT_SECTION_HEADER marking structure, structured TABLE grids with MERGED_CELL spans. What it doesn't provide: markdown, cross-page hierarchy, or retrieval units — and without the LAYOUT feature, not even reliable reading order. Details: The Optimal Chunker for AWS Textract.

pgvector adds a vector column type to Postgres, with HNSW and IVFFlat indexes and cosine, L2, and inner-product operators — plus everything Postgres already had: JOINs, transactions, tsvector full-text search, and no new infrastructure if your stack already runs on RDS or Aurora. What it doesn't provide: any opinion about what a row should contain. It finds nearest neighbors of whatever you embedded — including scrambled Block fragments, if that's what you inserted. Details: The Optimal Chunks for the Best Retrieval in pgvector.

The naive wiring, and where it breaks ​

The common recipe — iterate LINE blocks, join with newlines, RecursiveCharacterTextSplitter, embed, INSERT INTO documents (content, embedding) — breaks in a pair-specific way:

  • Textract's reading order dies at the flatten. LINE blocks carry geometry, not sequence; position-sorting interleaves the columns of exactly the two-column reports Textract is bought for. Postgres then durably, transactionally commits prose no human wrote — the transaction guarantees protect scrambled text.
  • The relational advantage goes unused. A single flat documents table stores your corpus the way a blob store would. Page numbers and heading depth were destroyed before the INSERT, so the WHERE clauses you chose Postgres for — per-document scoping, page-cited answers — have nothing to filter on, and no JOIN can recover a fragment's lineage, because lineage was never a row.
  • Overlap bloats the index. Splitter overlap embeds every boundary span twice: duplicate rows, near-duplicate vectors in the HNSW graph, and top-k results where hits 2 and 3 restate hit 1.

The pipeline, end to end ​

bash
pip install boto3 poma "psycopg[binary]" pgvector sentence-transformers
python
import json
import os

import boto3
import psycopg
from pgvector.psycopg import register_vector
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer

# 1. Textract — your existing call, unchanged. LAYOUT is required.
textract = boto3.client("textract")
with open("contract.png", "rb") as f:
    response = textract.analyze_document(
        Document={"Bytes": f.read()},
        FeatureTypes=["LAYOUT", "TABLES"],
    )
with open("contract.textract.json", "w") as f:
    json.dump(response, f)

# 2. The missing link — raw Blocks[] in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.textract.json",  # Textract shape auto-detected
    download_dir="store",
    filename="contract.poma",  # keep the archive: its chunk records carry `page`
)
raw = unpack_poma_archive(poma_archive_path="store/contract.poma")

# 3. Postgres — two joined tables, ANN + full-text indexes.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
embeddings = model.encode([cs.to_embed for cs in result.chunksets])

DDL = """
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE IF NOT EXISTS chunks (
    file_id     text NOT NULL,
    chunk_index int  NOT NULL,
    depth       int  NOT NULL,
    page        int,
    content     text NOT NULL,
    PRIMARY KEY (file_id, chunk_index)
);
CREATE TABLE IF NOT EXISTS chunksets (
    id            bigserial PRIMARY KEY,
    file_id       text  NOT NULL,
    chunk_indices int[] NOT NULL,
    to_embed      text  NOT NULL,
    embedding     vector(384) NOT NULL,
    tsv           tsvector GENERATED ALWAYS AS (to_tsvector('english', to_embed)) STORED
);
CREATE INDEX IF NOT EXISTS chunksets_ann ON chunksets USING hnsw (embedding vector_cosine_ops);
CREATE INDEX IF NOT EXISTS chunksets_fts ON chunksets USING gin (tsv);
"""

with psycopg.connect(os.environ["DATABASE_URL"]) as conn:
    register_vector(conn)
    conn.execute(DDL)
    with conn.transaction():  # document + chunks land atomically, or not at all
        for c in raw["chunks"]:
            conn.execute(
                "INSERT INTO chunks (file_id, chunk_index, depth, page, content)"
                " VALUES (%s, %s, %s, %s, %s)",
                (c["file_id"], c["chunk_index"], c["depth"], c.get("page"), c["content"]),
            )
        for cs, emb in zip(result.chunksets, embeddings):
            conn.execute(
                "INSERT INTO chunksets (file_id, chunk_indices, to_embed, embedding)"
                " VALUES (%s, %s, %s, %s)",
                (cs.file_id, cs.chunks, cs.to_embed, emb),  # cs.chunks is a list of chunk indices
            )

POMA validates the payload up front: a Textract result missing LAYOUT blocks fails with a clear 422 telling you to re-run with FeatureTypes including "LAYOUT" — never scrambled multi-column prose committed to your database. Tables are spliced by geometry and arrive as HTML with rowspan/colspan; figures (Textract returns no crops) are counted as offloaded content in content_metadata, visible in the ingest report. To force or suppress detection, set external_ocr_source to "textract" or "none".

On our reference legal document, this pipeline answers the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss. Methodology: Document Ingestion & Chunking for RAG.

Retrieval: ANN and lineage in one SQL round trip ​

This is the pgvector-specific payoff. Standalone vector stores return point payloads; Postgres returns the hit and its ancestry in the same statement:

sql
WITH hits AS (
    SELECT id, file_id, chunk_indices,
           embedding <=> %(query_vec)s AS distance
    FROM chunksets
    WHERE file_id = %(file_id)s        -- relational scoping, plain WHERE clause
    ORDER BY embedding <=> %(query_vec)s
    LIMIT 3
)
SELECT h.id AS chunkset_id, h.distance, c.depth, c.page, c.content
FROM hits h
JOIN chunks c
  ON c.file_id = h.file_id
 AND c.chunk_index = ANY (h.chunk_indices)
ORDER BY h.distance, c.chunk_index;

Deduplicate the returned chunk rows across the retrieved chunksets and merge them in chunk_index order — that is cheatsheet assembly, and it is what keeps retrieved context at a fraction of splitter output. For hybrid retrieval, add a second CTE that matches tsv @@ websearch_to_tsquery('english', %(query_text)s) and union or rerank: exact tokens (invoice numbers, clause references, form values — the substance of Textract-processed documents) hit through full-text search while paraphrase queries hit through the embedding.

Metadata mapping: POMA fields → Postgres primitives ​

POMA chunk fieldPostgres / pgvector primitiveWhat it enables
to_embedvector(384) column + HNSW (vector_cosine_ops); generated tsvector column + GINhybrid paraphrase + exact-term retrieval
file_idtext column, primary-key componentscope queries to one document with a WHERE clause
pageint columnpage-cited answers, page-range filters
depthint columnfilter or re-rank by hierarchy level
chunk_indexint column, primary-key componentstable ordering at assembly time
chunkset lineagechunk_indices int[] + JOIN chunksfull root-to-leaf path in the same query as the ANN search

Frequently asked questions ​

How do I get AWS Textract results into pgvector for RAG? ​

Run AnalyzeDocument with the LAYOUT feature, save the raw JSON, and run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt). Embed each chunkset's to_embed, insert chunks and chunksets as two joined tables in one transaction, and retrieve with a single ANN + JOIN statement.

What Postgres schema should Textract chunks use with pgvector? ​

A chunks table keyed by (file_id, chunk_index) with content, depth, page; a chunksets table with file_id, chunk_indices int[], to_embed, a vector embedding, and a generated tsvector. HNSW index on the embedding, GIN on the tsvector.

Yes — put the ANN search in a CTE, then JOIN the chunks table on file_id and the member chunk_indices. One round trip returns the hits plus every ancestor chunk on their root-to-leaf paths, ready for cheatsheet assembly.

Should I combine pgvector with tsvector full-text search for Textract documents? ​

Yes. Textract-processed documents are dense with exact tokens — invoice numbers, clause references, form values — that embeddings blur. A generated tsvector column is one line of DDL and gives you hybrid retrieval in the same SQL statement.

Do I need a separate vector database if my Textract stack already runs on RDS Postgres? ​

No. pgvector gives the Postgres you already operate HNSW/IVFFlat ANN search plus transactional ingest. The missing piece is what you embed: hierarchy-preserving chunksets, not flattened Block fragments.

Same parser, different store: Textract → Qdrant · Textract → Milvus · Textract → Chroma

Same store, different parser: Mistral OCR → pgvector · Azure Document Intelligence → pgvector · Docling → pgvector

Foundations: The Optimal Chunker for AWS Textract · The Optimal Chunks for pgvector · All pipeline recipes · RAG chunking guide