LlamaParse to pgvector: Chunking for RAG
The short answer: LlamaParse gives you excellent per-page markdown; pgvector gives you vector search inside the relational engine you already run. Wired together naively — concatenate, split, embed, one flat table — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy, and the flat table wastes the one thing Postgres does better than any dedicated vector store: relations. The missing link is POMA: PrimeCut().ingest() consumes the raw LlamaParse result JSON (auto-detected), emits hierarchy-preserving chunksets, and those land in Postgres as two joined tables — so a single SQL statement runs the ANN search and reconstructs each hit's full heading lineage.
What each end of the pipeline actually provides
LlamaParse (LlamaIndex's parser) returns pages[], each with a page number, an md string (markdown — headings marked, tables inline), a plain text flattening, and an images list whose bytes stay server-side. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for LlamaParse.
pgvector provides a vector column type, HNSW and IVFFlat indexes, cosine/L2/inner-product operators — and everything else Postgres has: JOINs, B-tree and GIN indexes, tsvector full-text search, transactions. What it doesn't provide: any opinion about what a row should contain. It ranks nearest neighbors of whatever you embedded — including context-free fragments, if that's what you inserted. Details: The Optimal Chunks for the Best Retrieval in pgvector.
The naive wiring, and where it breaks
The common recipe — "\n".join(p["md"] for p in pages) → RecursiveCharacterTextSplitter → embed → INSERT INTO embeddings (content, embedding) — breaks in pair-specific ways:
- LlamaParse's page numbers vanish at the join, so there is no
pagecolumn to index, filter, or cite — the parser handed you per-page structure and the pipeline threw it away before the firstINSERT. - Dead image references become lexemes. LlamaParse keeps image bytes server-side, so the saved JSON carries only
links. Split naively, those land in your rows — and the moment you add a generatedtsvectorfor hybrid search,to_tsvectorindexesimg_p3_2.pngas a lexeme. The keyword lane of your hybrid query now matches figure filenames. - One flat table wastes the engine. Fragments in
embeddings(id, content, embedding)have no chunks/chunksets/membership relations, so pgvector's structural advantage — JOINing full lineage in the same statement as the ANN search — is simply unavailable. - Overlap bloats the table. Splitter overlap embeds every boundary span twice: more rows, a slower HNSW build, and top-k results where hits 2 and 3 are near-duplicates of hit 1.
The pipeline, end to end
pip install llama-parse poma "psycopg[binary]" pgvector sentence-transformersThe schema — chunks and chunksets as two joined tables, exactly the shape PrimeCut emits:
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (
id bigserial PRIMARY KEY,
file_id text NOT NULL,
chunk_index int NOT NULL,
depth int NOT NULL, -- hierarchy level: 0 = title, deeper = finer
page int, -- from LlamaParse pages[].page, kept per chunk
content text NOT NULL,
UNIQUE (file_id, chunk_index)
);
CREATE TABLE chunksets (
id bigserial PRIMARY KEY,
file_id text NOT NULL,
to_embed text NOT NULL,
embedding vector(384) NOT NULL,
tsv tsvector GENERATED ALWAYS AS (to_tsvector('english', to_embed)) STORED
);
CREATE TABLE chunkset_members (
chunkset_id bigint NOT NULL REFERENCES chunksets(id) ON DELETE CASCADE,
chunk_id bigint NOT NULL REFERENCES chunks(id) ON DELETE CASCADE,
position int NOT NULL,
PRIMARY KEY (chunkset_id, chunk_id)
);
CREATE INDEX ON chunksets USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON chunksets USING gin (tsv);
CREATE INDEX ON chunks (file_id, chunk_index);Ingest — parse, chunk, and land everything atomically in one transaction:
import json
import os
import psycopg
from llama_parse import LlamaParse
from pgvector.psycopg import register_vector
from sentence_transformers import SentenceTransformer
from poma import PrimeCut
from poma.utils import unpack_poma_archive
# 1. LlamaParse — your existing call, unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
json.dump(json_result[0], f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.llamaparse.json", # LlamaParse shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive: its chunk records carry `page`
)
raw = unpack_poma_archive(poma_archive_path="store/contract.poma")
# 3. Postgres — document, chunks, and lineage land atomically.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2") # 384-dim
with psycopg.connect(os.environ["DATABASE_URL"]) as conn:
register_vector(conn)
with conn.transaction():
chunk_ids = {}
for c in raw["chunks"]:
row = conn.execute(
"INSERT INTO chunks (file_id, chunk_index, depth, page, content) "
"VALUES (%s, %s, %s, %s, %s) RETURNING id",
(c["file_id"], c["chunk_index"], c["depth"], c.get("page"), c["content"]),
).fetchone()
chunk_ids[c["chunk_index"]] = row[0]
for cs in result.chunksets:
row = conn.execute(
"INSERT INTO chunksets (file_id, to_embed, embedding) "
"VALUES (%s, %s, %s) RETURNING id",
(cs.file_id, cs.to_embed, model.encode(cs.to_embed)),
).fetchone()
for position, chunk_index in enumerate(cs.chunks):
conn.execute(
"INSERT INTO chunkset_members (chunkset_id, chunk_id, position) "
"VALUES (%s, %s, %s)",
(row[0], chunk_ids[chunk_index], position),
)Retrieval — ANN search and full lineage reconstruction in a single statement:
WITH hits AS (
SELECT id, embedding <=> %(query_vec)s AS distance
FROM chunksets
WHERE file_id = %(file_id)s -- ordinary indexed predicate
ORDER BY embedding <=> %(query_vec)s
LIMIT 5
)
SELECT h.id AS chunkset_id, h.distance, m.position, c.depth, c.page, c.content
FROM hits h
JOIN chunkset_members m ON m.chunkset_id = h.id
JOIN chunks c ON c.id = m.chunk_id
ORDER BY h.distance, m.position;POMA validates the payload up front (a corrupted or mislabeled upload 422s immediately), reads md over text so no structure is lost, neutralizes the dead  image references and counts them in content_metadata, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "llamaparse" or "none". Merging retrieved rows into deduplicated cheatsheets answered our reference legal-document benchmark with 337 tokens of context instead of 1,542 for a recursive-splitter baseline, zero information loss — methodology.
Metadata mapping: POMA fields → Postgres primitives
| POMA chunk field | Postgres primitive | What it enables |
|---|---|---|
to_embed | vector(384) column, HNSW-indexed + generated tsvector, GIN-indexed | dense ANN + keyword hybrid in one engine |
file_id | text column, B-tree indexed | scope the ANN query to one document with WHERE |
page (from LlamaParse pages[].page) | int column on chunks | page-cited answers, page-range predicates |
depth | int column on chunks | filter/re-rank by hierarchy level |
chunk_index | int column, UNIQUE (file_id, chunk_index) | stable ordering, idempotent re-ingest |
| chunkset lineage | chunkset_members join table | full root-to-leaf ancestry in the same query as the ANN search |
Frequently asked questions
How do I get LlamaParse results into pgvector for RAG?
Save the raw JSON from get_json_result, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), then insert in one transaction: chunk rows, embedded chunksets, and the membership table. Retrieval is one SQL statement — ANN in a CTE, JOIN for lineage.
What Postgres schema should I use for LlamaParse chunks?
Two joined tables plus a membership relation: chunks (file_id, chunk_index, depth, page, content) and chunksets (to_embed, vector column, generated tsvector), linked by chunkset_members. HNSW on the embedding, GIN on the tsvector, B-tree on (file_id, chunk_index).
Can I retrieve a LlamaParse document's heading hierarchy in the same query as the vector search?
Yes — put the ANN search in a CTE, then JOIN through chunkset_members to chunks. One round trip returns each hit with every ancestor heading, page, and depth. That JOIN is pgvector's structural advantage, and it only exists if ingest preserved the hierarchy as relations.
How do I combine vector and full-text search over LlamaParse output in Postgres?
Generated tsvector over to_embed, GIN index, reciprocal rank fusion in SQL. Neutralize LlamaParse's dead  refs first (POMA does this automatically) — otherwise image filenames become lexemes in the keyword lane.
What happens to LlamaParse image references in a pgvector pipeline?
The bytes live server-side at LlamaParse; the saved JSON has only  links. POMA neutralizes each one and counts it in content_metadata, so no dead filename reaches your embeddings or tsvector — loss is visible and quantified, never silent.
Related recipes
Same parser, different store: LlamaParse → Qdrant · LlamaParse → Pinecone · LlamaParse → Weaviate
Same store, different parser: Mistral OCR → pgvector · Docling → pgvector · Textract → pgvector
Foundations: The Optimal Chunker for LlamaParse · The Optimal Chunks for pgvector · All pipeline recipes · RAG chunking guide