Docling to pgvector: Chunking for RAG
The short answer: Docling gives you a typed document tree with explicit heading levels; pgvector gives you ANN search inside the relational engine you already run. Wired together naively — flatten to markdown, split, embed into one flat table — you throw away exactly the part each side is best at. The missing link is POMA: PrimeCut().ingest() consumes the saved DoclingDocument JSON (auto-detected), emits hierarchy-preserving chunksets, and those land in Postgres as what they naturally are — two joined tables, so one SQL statement runs the vector search and reconstructs the full document lineage.
What each end of the pipeline actually provides
Docling (IBM's open-source converter, also served via docling-serve) returns the DoclingDocument: a typed texts/tables/pictures/groups tree in which section_header items carry an explicit numeric level, tables are cell grids rather than pipe art, and repeating page furniture is pre-isolated in the furniture group with page_header/page_footer labels. What it doesn't provide: retrieval units. Its HybridChunker packs the tree into token windows — length control, not context. Details: The Optimal Chunker for Docling.
pgvector provides the vector column type, HNSW and IVFFlat indexes, distance operators that drop into ORDER BY, tsvector full-text search in the same engine, and transactions — all inside the Postgres you already operate. What it doesn't provide: any opinion about rows. It orders whatever you inserted by whatever distance you ask for. Details: The Optimal Chunks for the Best Retrieval in pgvector.
The naive wiring, and where it breaks
The common recipe — export_to_markdown() → RecursiveCharacterTextSplitter → embed → one flat documents(text, embedding) table — breaks in a pair-specific way: you are running a relational database and storing zero relations.
- Docling's explicit
section_headerlevels are parent–child data — precisely what Postgres was built to store andJOIN. Flattening reduces them to#glyphs before the firstINSERT, so there is nodepthcolumn to filter on and no lineage to reconstruct. The one store in this series that could hold the hierarchy as rows gets handed a string. - Page provenance dies at flattening, so the schema has no
pagecolumn, and page-cited answers or indexed page-range predicates are off the table. - Splitter overlap embeds every boundary span twice — a fatter
chunksetstable, a slower HNSW build, andLIMIT 5results that repeat each other. - Multi-step ingest scripts without a transaction leave a half-ingested corpus after a crash — in the one store that offers atomicity for free.
The pipeline, end to end
pip install docling poma "psycopg[binary]" pgvectorThe schema stores POMA's output as what it is — a hierarchy:
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (
id bigserial PRIMARY KEY,
file_id text NOT NULL,
chunk_index int NOT NULL,
depth int NOT NULL, -- from Docling's explicit section_header levels
page int,
content text NOT NULL, -- tables arrive as HTML, never cut mid-row
UNIQUE (file_id, chunk_index)
);
CREATE TABLE chunksets (
id bigserial PRIMARY KEY,
file_id text NOT NULL,
to_embed text NOT NULL,
embedding vector(384) NOT NULL,
tsv tsvector GENERATED ALWAYS AS (to_tsvector('english', to_embed)) STORED
);
CREATE TABLE chunkset_members (
chunkset_id bigint NOT NULL REFERENCES chunksets(id) ON DELETE CASCADE,
chunk_id bigint NOT NULL REFERENCES chunks(id) ON DELETE CASCADE,
position int NOT NULL, -- 0 = document root … last = the leaf
PRIMARY KEY (chunkset_id, chunk_id)
);
CREATE INDEX ON chunksets USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON chunksets USING gin (tsv);Ingest is one transaction — the document and every chunk land atomically, or not at all:
import json
import os
import psycopg
from docling.document_converter import DocumentConverter
from pgvector.psycopg import register_vector
from poma import PrimeCut
from poma.utils import unpack_poma_archive
# 1. Docling — your existing conversion, unchanged.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
json.dump(doc.export_to_dict(), f)
# 2. The missing link — DoclingDocument tree in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.docling.json", # Docling shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive: its chunk records carry `page`
)
raw = unpack_poma_archive(poma_archive_path="store/contract.poma")
# 3. Postgres — hierarchy as rows, atomically.
with psycopg.connect(os.environ["DATABASE_URL"]) as conn:
register_vector(conn)
with conn.transaction():
chunk_ids = {}
for c in raw["chunks"]:
row = conn.execute(
"INSERT INTO chunks (file_id, chunk_index, depth, page, content) "
"VALUES (%s, %s, %s, %s, %s) RETURNING id",
(c["file_id"], c["chunk_index"], c["depth"], c.get("page"), c["content"]),
).fetchone()
chunk_ids[c["chunk_index"]] = row[0]
for cs in result.chunksets:
row = conn.execute(
"INSERT INTO chunksets (file_id, to_embed, embedding) "
"VALUES (%s, %s, %s) RETURNING id",
(cs.file_id, cs.to_embed, embed(cs.to_embed)), # your embedding model
).fetchone()
for position, chunk_index in enumerate(cs.chunks):
conn.execute(
"INSERT INTO chunkset_members (chunkset_id, chunk_id, position) "
"VALUES (%s, %s, %s)",
(row[0], chunk_ids[chunk_index], position),
)Retrieval is where the pairing pays off — ANN search and full lineage reconstruction in a single statement:
WITH hits AS (
SELECT id, embedding <=> %(query_vec)s AS distance
FROM chunksets
WHERE file_id = %(file_id)s -- ordinary indexed predicate
ORDER BY embedding <=> %(query_vec)s
LIMIT 5
)
SELECT h.distance, m.position, c.depth, c.page, c.content
FROM hits h
JOIN chunkset_members m ON m.chunkset_id = h.id
JOIN chunks c ON c.id = m.chunk_id
ORDER BY h.distance, m.position;POMA validates the payload up front (a corrupted or mislabeled upload 422s immediately, never a silently degraded index), honors the furniture Docling already isolated, keeps tables as HTML, and accepts docling-serve md_content envelopes via a markdown passthrough. To force or suppress detection, set external_ocr_source to "docling" or "none". The returned rows merge into a deduplicated, prompt-ready cheatsheet — on the reference legal-document benchmark, 337 tokens of context versus 1,542 for a recursive-splitter baseline, with zero information loss (methodology).
Metadata mapping: POMA fields → pgvector primitives
| POMA chunk field | Postgres primitive | What it enables |
|---|---|---|
to_embed | vector(384) column (HNSW) + generated tsvector (GIN) | dense ANN + keyword hybrid in one engine |
file_id | indexed text column on both tables | per-document scoping; the join key of the schema |
page | int column | page-cited answers, page-range predicates |
depth (from Docling section_header level) | int column | WHERE depth <= 2, structural re-ranking |
chunk_index | int column, UNIQUE (file_id, chunk_index) | stable ordering at assembly time |
| chunkset lineage | chunkset_members join table | root-to-leaf lineage in the same query as ANN |
Frequently asked questions
How do I get Docling output into pgvector for RAG?
Save export_to_dict() as JSON, run PrimeCut().ingest() on it (auto-detected via schema_name), insert chunks and chunksets into two joined tables in one transaction, embed to_embed into a vector column with HNSW. Retrieval is one SQL statement: an ANN CTE joined back to chunks for full lineage.
What Postgres schema should I use for Docling chunks?
Three tables: chunks (file_id, chunk_index, depth, page, content), chunksets (to_embed, vector, generated tsvector), and chunkset_members linking them. Docling's explicit section_header levels become the depth integer — hierarchy as indexed columns, not flattened text.
Can I query Docling's heading levels with SQL in pgvector?
Yes. POMA writes each section_header item's explicit level to every chunk as depth. In an integer column, heading levels are ordinary SQL: WHERE depth <= 2 scopes to top-level sections, and a JOIN returns every ancestor of a matched chunkset alongside the vector search.
How do I combine vector search and keyword search for Docling documents in Postgres?
A generated tsvector column over to_embed, indexed with GIN, next to the HNSW index. Fuse the two ranked lists with reciprocal rank fusion in plain SQL. Exact tokens Docling preserved faithfully come from tsvector; paraphrases come from the embedding.
Do I need Docling's HybridChunker if I store chunks in pgvector?
No — token windows underuse Postgres. They control length but leave rows without machine-readable lineage. Chunksets keep hierarchy as relations: lineage is a JOIN, depth is a WHERE clause, and every retrieved unit is self-explanatory.
Related recipes
Same parser, different store: Docling → Qdrant · Docling → Milvus · Docling → Chroma
Same store, different parser: Mistral OCR → pgvector · Unstructured.io → pgvector · Textract → pgvector
Foundations: The Optimal Chunker for Docling · The Optimal Chunks for pgvector · All pipeline recipes · RAG chunking guide