Skip to content

LlamaParse to pgvector: Chunking for RAG ​

The short answer: LlamaParse gives you excellent per-page markdown; pgvector gives you vector search inside the relational engine you already run. Wired together naively — concatenate, split, embed, one flat table — they still produce mediocre RAG, because nothing in between rebuilds the document's hierarchy, and the flat table wastes the one thing Postgres does better than any dedicated vector store: relations. The missing link is POMA: PrimeCut().ingest() consumes the raw LlamaParse result JSON (auto-detected), emits hierarchy-preserving chunksets, and those land in Postgres as two joined tables — so a single SQL statement runs the ANN search and reconstructs each hit's full heading lineage.

What each end of the pipeline actually provides ​

LlamaParse (LlamaIndex's parser) returns pages[], each with a page number, an md string (markdown — headings marked, tables inline), a plain text flattening, and an images list whose bytes stay server-side. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for LlamaParse.

pgvector provides a vector column type, HNSW and IVFFlat indexes, cosine/L2/inner-product operators — and everything else Postgres has: JOINs, B-tree and GIN indexes, tsvector full-text search, transactions. What it doesn't provide: any opinion about what a row should contain. It ranks nearest neighbors of whatever you embedded — including context-free fragments, if that's what you inserted. Details: The Optimal Chunks for the Best Retrieval in pgvector.

The naive wiring, and where it breaks ​

The common recipe — "\n".join(p["md"] for p in pages) → RecursiveCharacterTextSplitter → embed → INSERT INTO embeddings (content, embedding) — breaks in pair-specific ways:

  • LlamaParse's page numbers vanish at the join, so there is no page column to index, filter, or cite — the parser handed you per-page structure and the pipeline threw it away before the first INSERT.
  • Dead image references become lexemes. LlamaParse keeps image bytes server-side, so the saved JSON carries only ![](name) links. Split naively, those land in your rows — and the moment you add a generated tsvector for hybrid search, to_tsvector indexes img_p3_2.png as a lexeme. The keyword lane of your hybrid query now matches figure filenames.
  • One flat table wastes the engine. Fragments in embeddings(id, content, embedding) have no chunks/chunksets/membership relations, so pgvector's structural advantage — JOINing full lineage in the same statement as the ANN search — is simply unavailable.
  • Overlap bloats the table. Splitter overlap embeds every boundary span twice: more rows, a slower HNSW build, and top-k results where hits 2 and 3 are near-duplicates of hit 1.

The pipeline, end to end ​

bash
pip install llama-parse poma "psycopg[binary]" pgvector sentence-transformers

The schema — chunks and chunksets as two joined tables, exactly the shape PrimeCut emits:

sql
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
    id          bigserial PRIMARY KEY,
    file_id     text NOT NULL,
    chunk_index int  NOT NULL,
    depth       int  NOT NULL,   -- hierarchy level: 0 = title, deeper = finer
    page        int,             -- from LlamaParse pages[].page, kept per chunk
    content     text NOT NULL,
    UNIQUE (file_id, chunk_index)
);

CREATE TABLE chunksets (
    id        bigserial PRIMARY KEY,
    file_id   text NOT NULL,
    to_embed  text NOT NULL,
    embedding vector(384) NOT NULL,
    tsv       tsvector GENERATED ALWAYS AS (to_tsvector('english', to_embed)) STORED
);

CREATE TABLE chunkset_members (
    chunkset_id bigint NOT NULL REFERENCES chunksets(id) ON DELETE CASCADE,
    chunk_id    bigint NOT NULL REFERENCES chunks(id)    ON DELETE CASCADE,
    position    int    NOT NULL,
    PRIMARY KEY (chunkset_id, chunk_id)
);

CREATE INDEX ON chunksets USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON chunksets USING gin (tsv);
CREATE INDEX ON chunks (file_id, chunk_index);

Ingest — parse, chunk, and land everything atomically in one transaction:

python
import json
import os

import psycopg
from llama_parse import LlamaParse
from pgvector.psycopg import register_vector
from sentence_transformers import SentenceTransformer

from poma import PrimeCut
from poma.utils import unpack_poma_archive

# 1. LlamaParse — your existing call, unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
    json.dump(json_result[0], f)

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.llamaparse.json",  # LlamaParse shape auto-detected
    download_dir="store",
    filename="contract.poma",  # keep the archive: its chunk records carry `page`
)
raw = unpack_poma_archive(poma_archive_path="store/contract.poma")

# 3. Postgres — document, chunks, and lineage land atomically.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")  # 384-dim

with psycopg.connect(os.environ["DATABASE_URL"]) as conn:
    register_vector(conn)
    with conn.transaction():
        chunk_ids = {}
        for c in raw["chunks"]:
            row = conn.execute(
                "INSERT INTO chunks (file_id, chunk_index, depth, page, content) "
                "VALUES (%s, %s, %s, %s, %s) RETURNING id",
                (c["file_id"], c["chunk_index"], c["depth"], c.get("page"), c["content"]),
            ).fetchone()
            chunk_ids[c["chunk_index"]] = row[0]

        for cs in result.chunksets:
            row = conn.execute(
                "INSERT INTO chunksets (file_id, to_embed, embedding) "
                "VALUES (%s, %s, %s) RETURNING id",
                (cs.file_id, cs.to_embed, model.encode(cs.to_embed)),
            ).fetchone()
            for position, chunk_index in enumerate(cs.chunks):
                conn.execute(
                    "INSERT INTO chunkset_members (chunkset_id, chunk_id, position) "
                    "VALUES (%s, %s, %s)",
                    (row[0], chunk_ids[chunk_index], position),
                )

Retrieval — ANN search and full lineage reconstruction in a single statement:

sql
WITH hits AS (
    SELECT id, embedding <=> %(query_vec)s AS distance
    FROM chunksets
    WHERE file_id = %(file_id)s          -- ordinary indexed predicate
    ORDER BY embedding <=> %(query_vec)s
    LIMIT 5
)
SELECT h.id AS chunkset_id, h.distance, m.position, c.depth, c.page, c.content
FROM hits h
JOIN chunkset_members m ON m.chunkset_id = h.id
JOIN chunks c           ON c.id = m.chunk_id
ORDER BY h.distance, m.position;

POMA validates the payload up front (a corrupted or mislabeled upload 422s immediately), reads md over text so no structure is lost, neutralizes the dead ![](name) image references and counts them in content_metadata, strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "llamaparse" or "none". Merging retrieved rows into deduplicated cheatsheets answered our reference legal-document benchmark with 337 tokens of context instead of 1,542 for a recursive-splitter baseline, zero information loss — methodology.

Metadata mapping: POMA fields → Postgres primitives ​

POMA chunk fieldPostgres primitiveWhat it enables
to_embedvector(384) column, HNSW-indexed + generated tsvector, GIN-indexeddense ANN + keyword hybrid in one engine
file_idtext column, B-tree indexedscope the ANN query to one document with WHERE
page (from LlamaParse pages[].page)int column on chunkspage-cited answers, page-range predicates
depthint column on chunksfilter/re-rank by hierarchy level
chunk_indexint column, UNIQUE (file_id, chunk_index)stable ordering, idempotent re-ingest
chunkset lineagechunkset_members join tablefull root-to-leaf ancestry in the same query as the ANN search

Frequently asked questions ​

How do I get LlamaParse results into pgvector for RAG? ​

Save the raw JSON from get_json_result, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), then insert in one transaction: chunk rows, embedded chunksets, and the membership table. Retrieval is one SQL statement — ANN in a CTE, JOIN for lineage.

What Postgres schema should I use for LlamaParse chunks? ​

Two joined tables plus a membership relation: chunks (file_id, chunk_index, depth, page, content) and chunksets (to_embed, vector column, generated tsvector), linked by chunkset_members. HNSW on the embedding, GIN on the tsvector, B-tree on (file_id, chunk_index).

Yes — put the ANN search in a CTE, then JOIN through chunkset_members to chunks. One round trip returns each hit with every ancestor heading, page, and depth. That JOIN is pgvector's structural advantage, and it only exists if ingest preserved the hierarchy as relations.

How do I combine vector and full-text search over LlamaParse output in Postgres? ​

Generated tsvector over to_embed, GIN index, reciprocal rank fusion in SQL. Neutralize LlamaParse's dead ![](name) refs first (POMA does this automatically) — otherwise image filenames become lexemes in the keyword lane.

What happens to LlamaParse image references in a pgvector pipeline? ​

The bytes live server-side at LlamaParse; the saved JSON has only ![](name) links. POMA neutralizes each one and counts it in content_metadata, so no dead filename reaches your embeddings or tsvector — loss is visible and quantified, never silent.

Same parser, different store: LlamaParse → Qdrant · LlamaParse → Pinecone · LlamaParse → Weaviate

Same store, different parser: Mistral OCR → pgvector · Docling → pgvector · Textract → pgvector

Foundations: The Optimal Chunker for LlamaParse · The Optimal Chunks for pgvector · All pipeline recipes · RAG chunking guide