Azure Document Intelligence to pgvector: Chunking for RAG
The short answer: Azure Document Intelligence gives you an excellent layout analysis; pgvector gives you ANN search inside the database you already run. Wired together naively — flatten, split, embed, insert into one table — they still produce mediocre RAG, because the structure the layout model recovered never reaches the relational engine built to hold it. The missing link is POMA: PrimeCut().ingest() consumes the raw analyze result (auto-detected, markdown or JSON mode), emits hierarchy-preserving chunksets, and those land in Postgres as two joined tables — so one SQL statement runs the vector search and reconstructs each hit's full lineage.
What each end of the pipeline actually provides
Azure Document Intelligence (prebuilt-layout) returns one of two shapes: markdown mode — a single reading-order markdown string with inline HTML tables, PageBreak delimiters, and page-furniture comments — or JSON mode, paragraphs[] ordered by span offset with role tags (title, sectionHeading, …) plus tables[] as cell grids. Multi-column de-interleaving happens server-side. What it doesn't provide: cross-page hierarchy (roles classify paragraphs, they don't nest them), retrieval units, or figure bytes. Details: The Optimal Chunker for Azure Document Intelligence.
pgvector provides the vector column type, HNSW and IVFFlat indexes, distance operators that drop into ORDER BY, tsvector full-text search in the same engine, and transactions. What it doesn't provide: any opinion about what a row should contain. It orders rows by whatever distance you ask for, over whatever you inserted. Details: The Optimal Chunks for the Best Retrieval in pgvector.
The naive wiring, and where it breaks
The common recipe — take content (or flatten paragraphs[] to text), run a recursive character splitter, embed, INSERT into a single (id, text, embedding) table — breaks in a pair-specific way:
- Azure's structure is flattened at the door of the one store that could model it. Span-ordered, role-tagged paragraphs collapse into an untyped text column.
rolecould have been adepthint,PageBreakdelimiters apagecolumn, the heading tree a join table — Postgres is a relational database, and the naive pipeline hands it a blob. - Page anchors vanish. In markdown mode the
PageBreakcomments are the only page signal; a character splitter destroys them, so no column can ever say which page a chunk came from, and page-cited answers become impossible. - Overlap bloats the HNSW index. Splitter overlap embeds every boundary span twice — a fatter table, a slower index build, and
LIMIT 5results where hits 2 and 3 are near-duplicates of hit 1. - Loop inserts without a transaction leave a half-ingested filing when the pipeline crashes mid-document — a silently partial corpus that Postgres would have prevented for free.
The pipeline, end to end
pip install azure-ai-documentintelligence poma "psycopg[binary]"The schema mirrors the document's real hierarchy — chunks and chunksets as two joined tables (full design discussion on the pgvector chunks page):
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (
id bigserial PRIMARY KEY,
file_id text NOT NULL,
chunk_index int NOT NULL,
depth int NOT NULL, -- from Azure roles + POMA's rebuilt heading tree
page int, -- from PageBreak splits or span offsets
content text NOT NULL, -- Azure's HTML tables arrive intact, never cut
UNIQUE (file_id, chunk_index)
);
CREATE TABLE chunksets (
id bigserial PRIMARY KEY,
file_id text NOT NULL,
to_embed text NOT NULL,
embedding vector(384) NOT NULL,
tsv tsvector GENERATED ALWAYS AS (to_tsvector('english', to_embed)) STORED
);
CREATE TABLE chunkset_members (
chunkset_id bigint NOT NULL REFERENCES chunksets(id) ON DELETE CASCADE,
chunk_id bigint NOT NULL REFERENCES chunks(id) ON DELETE CASCADE,
position int NOT NULL, -- 0 = document root … last = the leaf
PRIMARY KEY (chunkset_id, chunk_id)
);
CREATE INDEX ON chunksets USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON chunksets USING gin (tsv);import json
import os
import psycopg
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.core.credentials import AzureKeyCredential
from poma import PrimeCut
from poma.utils import unpack_poma_archive
# 1. Your existing Azure Document Intelligence call — unchanged.
di = DocumentIntelligenceClient(
endpoint=os.environ["AZURE_DI_ENDPOINT"],
credential=AzureKeyCredential(os.environ["AZURE_DI_KEY"]),
)
with open("contract.pdf", "rb") as f:
poller = di.begin_analyze_document(
"prebuilt-layout",
body=f,
output_content_format="markdown", # omit for classic JSON mode — both work
)
with open("contract.azure-di.json", "w") as f:
json.dump(poller.result().as_dict(), f)
# 2. The missing link — raw analyze result in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.azure-di.json", # Azure shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive: its chunk records carry `page`
)
raw = unpack_poma_archive(poma_archive_path="store/contract.poma")
# 3. One transaction: the document lands atomically, or not at all.
with psycopg.connect() as conn, conn.transaction():
chunk_ids = {}
for c in raw["chunks"]:
row = conn.execute(
"INSERT INTO chunks (file_id, chunk_index, depth, page, content) "
"VALUES (%s, %s, %s, %s, %s) RETURNING id",
(c["file_id"], c["chunk_index"], c["depth"], c.get("page"), c["content"]),
).fetchone()
chunk_ids[c["chunk_index"]] = row[0]
for cs in result.chunksets:
row = conn.execute(
"INSERT INTO chunksets (file_id, to_embed, embedding) "
"VALUES (%s, %s, %s) RETURNING id",
(cs.file_id, cs.to_embed, embed(cs.to_embed)), # your embedding model
).fetchone()
for position, chunk_index in enumerate(cs.chunks):
conn.execute(
"INSERT INTO chunkset_members (chunkset_id, chunk_id, position) "
"VALUES (%s, %s, %s)",
(row[0], chunk_ids[chunk_index], position),
)Retrieval is where the pairing pays off — ANN search and full lineage reconstruction in a single round trip:
WITH hits AS (
SELECT id, embedding <=> %(query_vec)s AS distance
FROM chunksets
WHERE file_id = %(file_id)s -- ordinary indexed predicate
ORDER BY embedding <=> %(query_vec)s
LIMIT 5
)
SELECT h.distance, m.position, c.depth, c.page, c.content
FROM hits h
JOIN chunkset_members m ON m.chunkset_id = h.id
JOIN chunks c ON c.id = m.chunk_id
ORDER BY h.distance, m.position;Each hit comes back as ordered rows from root to leaf — every ancestor heading with its page and depth — the raw material for cheatsheet assembly: deduplicate the shared lineage across hits and merge into one prompt-ready block. On our reference legal-document benchmark that meant 337 tokens of context versus 1,542 for a recursive-splitter baseline, with zero information loss — methodology here.
POMA validates the payload up front (a corrupted or mislabeled upload 422s immediately, never a silently degraded index), handles both Azure shapes, drops the PageHeader/PageFooter/PageNumber furniture, keeps HTML tables whole, and counts Azure's byte-less figures as offloaded content in content_metadata — visible loss, never silent. To force or suppress detection, set external_ocr_source to "azure_di" or "none".
Metadata mapping: POMA fields → Postgres primitives
| POMA chunk field | Postgres primitive | What it enables |
|---|---|---|
to_embed | vector(384) column + generated tsvector (GIN) | ANN + keyword hybrid in one engine |
file_id | text column, indexed WHERE predicate | scope queries to one document |
page (from PageBreak / span offsets) | int column | page-cited answers, BETWEEN range filters |
depth (from Azure roles + rebuilt tree) | int column | WHERE depth <= 2 structural filters |
chunk_index | UNIQUE (file_id, chunk_index) | stable ordering at assembly time |
| chunkset lineage | chunkset_members join table | full ancestry in the same query as the ANN search |
Frequently asked questions
How do I get Azure Document Intelligence output into pgvector for RAG?
Save the raw analyze result JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), then insert chunks and chunksets as two joined tables in one transaction, embedding each chunkset's to_embed into a vector column. Retrieval is one SQL statement: ANN search plus a lineage JOIN.
What Postgres schema fits Azure Document Intelligence results?
chunks (file_id, chunk_index, depth, page, content) and chunksets (to_embed, vector column, generated tsvector), linked by a membership table. The structure prebuilt-layout recovered becomes typed, indexed columns instead of one flattened text blob.
Can I keep Azure Document Intelligence page numbers queryable in Postgres?
Yes — POMA recovers page from the PageBreak comments (markdown mode) or span offsets (JSON mode) and records it on every chunk record in the .poma archive; the SDK's PomaChunk objects expose depth but not page, so read the archive with unpack_poma_archive. As an int column it's an ordinary indexed predicate, and the lineage JOIN returns pages for citation. Flatten-and-split pipelines destroy this anchor.
Should I use tsvector hybrid search for Azure Document Intelligence content in pgvector?
Yes: contracts, filings, and forms are full of exact tokens (clause numbers, invoice IDs, defined terms) that dense embeddings blur. A generated tsvector over to_embed with a GIN index gives the keyword lane in the same engine; fuse with reciprocal rank fusion in plain SQL.
Why ingest Azure Document Intelligence chunks in a single Postgres transaction?
Because a half-ingested filing silently returns partial answers. Postgres makes atomicity free — wrap chunk, chunkset, and membership inserts in one transaction and the document lands whole or not at all, a guarantee standalone vector stores generally can't make across related records.
Related recipes
Same parser, different store: Azure Document Intelligence → Qdrant · Azure Document Intelligence → Milvus · Azure Document Intelligence → Weaviate
Same store, different parser: Mistral OCR → pgvector · Textract → pgvector · Docling → pgvector
Foundations: The Optimal Chunker for Azure Document Intelligence · The Optimal Chunks for pgvector · All pipeline recipes · RAG chunking guide