Source: http://www.poma-ai.com/docs/optimal-chunks-pgvector

# The Optimal Chunks for the Best Retrieval in pgvector

<ByAuthor />

**The short answer:** the optimal chunks for pgvector are **chunksets** — self-explanatory units that carry their full heading lineage — stored as **two joined tables**: `chunks` for every typed unit and `chunksets` for the embedded retrieval units, with `file_id`, `page`, and `depth` as plain indexed columns. Because pgvector lives inside Postgres, one SQL statement can run the ANN search *and* JOIN back the full document lineage, something standalone vector stores need a second round trip for. POMA's PrimeCut emits exactly this relational shape from any document.

This page covers what pgvector does well, what it cannot do for you, and the schema plus query design that gets the most out of it.

## pgvector puts vector search inside the database you already run

For teams already on Postgres, pgvector is the lowest-friction path to RAG — no new infrastructure, no second system of record:

- **The `vector` column type** — embeddings live next to your relational data, governed by the same backups, roles, and replication.
- **HNSW and IVFFlat indexes** — approximate nearest-neighbor search at scale, chosen per table to trade build cost against recall.
- **Distance operators** — `<=>` (cosine distance), `<->` (L2), `<#>` (inner product) drop straight into `ORDER BY`.
- **`tsvector` in the same engine** — Postgres full-text search gives you the keyword half of hybrid retrieval without another service.
- **Transactions** — a document and all of its chunks land atomically, or not at all. No half-ingested corpus after a crashed pipeline.

None of these primitives, however, know anything about your documents. Postgres orders rows by whatever distance you ask for, over whatever vectors you inserted. If what you embedded is a context-free fragment — a paragraph cut loose from the section that gives it meaning — pgvector will faithfully return context-free fragments, in perfectly correct distance order. **The ceiling on retrieval quality is set before the first `INSERT`.**

## What "optimal chunks" means for pgvector, concretely

Four properties separate a Postgres RAG schema that answers questions from one that returns trivia:

1. **Every retrieved row is self-explanatory.** A chunkset is a root-to-leaf path through the document — `Master Services Agreement → Termination clauses → Early termination → "Either party may…"`. Retrieved alone, it still tells the LLM (and the human reading citations) exactly where it sits. A bare 512-token window does not. See [POMA chunksets](/learn/chunking/chunksets).
2. **The hierarchy is relational, not stringly.** This is pgvector's unique advantage: `chunks` and `chunksets` are naturally two tables joined by a membership relation. Where other stores flatten lineage into a metadata blob, Postgres keeps it as rows — so a `JOIN` reconstructs the full ancestry *in the same query as the ANN search*, and `WHERE file_id = …` or `depth <= 2` are ordinary indexed predicates.
3. **No overlap, no near-duplicates.** Overlap embeds every overlapped span twice: a fatter table, a slower HNSW build, and top-k results that repeat each other. Chunksets carry context structurally, so overlap is simply unnecessary and your `LIMIT 5` returns five *different* candidates.
4. **Hybrid with `tsvector`.** Legal, financial, and technical corpora are full of exact tokens (§ numbers, SKUs, defined terms) that dense embeddings smear. A generated `tsvector` column with a GIN index gives you the keyword lane in the same engine — fuse the two ranked lists with reciprocal rank fusion in plain SQL.

## The schema: chunks and chunksets as two joined tables

PrimeCut turns any document — PDF, DOCX, HTML, or a bring-your-own-OCR result JSON — into typed chunks and chunksets. In Postgres, that output maps onto a schema you would design anyway:

```sql
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
    id          bigserial PRIMARY KEY,
    file_id     text NOT NULL,
    chunk_index int  NOT NULL,
    depth       int  NOT NULL,   -- hierarchy level: 0 = title, deeper = finer
    page        int,
    content     text NOT NULL,   -- tables arrive as HTML, never cut mid-row
    UNIQUE (file_id, chunk_index)
);

CREATE TABLE chunksets (
    id        bigserial PRIMARY KEY,
    file_id   text NOT NULL,
    to_embed  text NOT NULL,     -- normalized text of the root-to-leaf unit
    embedding vector(384) NOT NULL,
    tsv       tsvector GENERATED ALWAYS AS (to_tsvector('english', to_embed)) STORED
);

CREATE TABLE chunkset_members (
    chunkset_id bigint NOT NULL REFERENCES chunksets(id) ON DELETE CASCADE,
    chunk_id    bigint NOT NULL REFERENCES chunks(id)    ON DELETE CASCADE,
    position    int    NOT NULL, -- 0 = document root … last = the leaf
    PRIMARY KEY (chunkset_id, chunk_id)
);

CREATE INDEX ON chunksets USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON chunksets USING gin (tsv);
CREATE INDEX ON chunks (file_id, chunk_index);
```

Ingest is one transaction — the document and every chunk land atomically:

```python
import psycopg
from poma import PrimeCut

client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.pdf")

with psycopg.connect() as conn, conn.transaction():
    chunk_ids = {}
    for chunk in result.chunks:
        row = conn.execute(
            "INSERT INTO chunks (file_id, chunk_index, depth, page, content) "
            "VALUES (%s, %s, %s, %s, %s) RETURNING id",
            (chunk.file_id, chunk.chunk_index, chunk.depth, chunk.page, chunk.content),
        ).fetchone()
        chunk_ids[chunk.chunk_index] = row[0]

    for cs in result.chunksets:
        row = conn.execute(
            "INSERT INTO chunksets (file_id, to_embed, embedding) "
            "VALUES (%s, %s, %s) RETURNING id",
            (cs.file_id, cs.to_embed, embed(cs.to_embed)),  # your embedding model
        ).fetchone()
        for position, chunk_index in enumerate(cs.chunks):
            conn.execute(
                "INSERT INTO chunkset_members (chunkset_id, chunk_id, position) "
                "VALUES (%s, %s, %s)",
                (row[0], chunk_ids[chunk_index], position),
            )
```

And retrieval is where the relational design pays off — ANN search and full lineage reconstruction in a single statement:

```sql
WITH hits AS (
    SELECT id, file_id, embedding <=> %(query_vec)s AS distance
    FROM chunksets
    WHERE file_id = %(file_id)s          -- ordinary indexed predicate
    ORDER BY embedding <=> %(query_vec)s
    LIMIT 5
)
SELECT h.id AS chunkset_id,
       h.distance,
       m.position,
       c.depth,
       c.page,
       c.content
FROM hits h
JOIN chunkset_members m ON m.chunkset_id = h.id
JOIN chunks c           ON c.id = m.chunk_id
ORDER BY h.distance, m.position;
```

Each hit comes back as ordered rows from root to leaf: every ancestor heading, its page, its depth. Feed those rows to POMA's cheatsheet assembly — retrieved chunksets share ancestors, and merging deduplicates the lineage into one coherent, prompt-ready block. On our reference legal-document benchmark that meant **337 tokens** of context versus **1,542** for a recursive-splitter baseline, with zero information loss — [methodology here](/document-ingestion-chunking-rag).

## Chunk shapes in pgvector, compared

| Property | Fixed 512-token chunks + overlap | Semantic chunks | **POMA chunksets** |
| --- | --- | --- | --- |
| Row is self-explanatory when retrieved alone | ✗ | Sometimes | **✓ (by construction)** |
| Near-duplicate vectors in the HNSW/IVFFlat index | Many (overlap) | Few | **None** |
| Hierarchy queryable with SQL (`JOIN`, `WHERE depth…`) | ✗ (flat text) | ✗ | **✓ (two joined tables)** |
| Lineage retrieved in the same query as ANN search | ✗ | ✗ | **✓ (one `JOIN`)** |
| Tables survive embedding | ✗ (cut mid-row) | Fragile | **✓ (HTML, never cut)** |
| Hybrid with `tsvector` | DIY over fragments | DIY | **✓ (generated column on `to_embed`)** |
| Post-retrieval assembly | Concatenate raw hits | Concatenate raw hits | **Cheatsheets (deduped lineage)** |

## Frequently asked questions

### What is the optimal chunk size for pgvector?

There is no universal token count — the optimal unit is a chunk that is self-explanatory without its neighbors. Chunksets achieve that at any size, which is why they beat fixed 512-token windows: every row returned by an ANN query can be handed to the LLM alone and still be understood. If you must fix a size, start at 512 tokens — but the size knob cannot fix missing context.

### What Postgres schema should I use for RAG chunks?

Two joined tables: `chunks` (file_id, chunk_index, depth, page, content) and `chunksets` (to_embed, `vector` column), linked by a membership table. The document's real hierarchy becomes relational, so a single `JOIN` reconstructs a retrieved chunkset's full root-to-leaf lineage in the same query that runs the vector search.

### Should I use HNSW or IVFFlat in pgvector for RAG?

For most RAG workloads, HNSW: better recall at a given query latency, and it builds incrementally on an empty table. IVFFlat builds faster and uses less memory, but must be created after data exists and its recall depends on `lists`/`probes` tuning. Neither can compensate for context-free chunks.

### How do I combine pgvector with full-text search for hybrid RAG?

Add a generated `tsvector` column over the chunkset's `to_embed` text, index it with GIN, and run `to_tsquery` matching alongside the vector query. Fuse the two ranked lists — reciprocal rank fusion is expressible in plain SQL with two CTEs and a `FULL OUTER JOIN`.

### How does chunk overlap affect a pgvector index?

Every overlapped span is embedded twice: a fatter table, a slower HNSW build, and near-duplicate hits crowding out diverse results. Chunks that carry their hierarchy don't need overlap.

### Can I retrieve a chunk's document hierarchy in the same SQL query as the vector search?

Yes — put the ANN search in a CTE (`ORDER BY embedding <=> query_vector LIMIT k`), then `JOIN` to the membership and chunks tables. One round trip returns each hit with every ancestor heading, page, and depth — the raw material for cheatsheet assembly.

## First-party ingest: vektoria's push()

pgvector is one of only two databases (with Chroma) that has a real first-party `vektoria` ingest connector — one call handles chunking-to-vector in a content-free shape:

```python
from poma.vektoria import push, get_connector
from poma.embeddings import get_embedder

stats = push(
    "contract.poma",
    volume="s3://your-bucket/poma",
    connector=get_connector("pgvector", dsn=os.environ["POSTGRES_DSN"], table="poma_chunksets"),
    embedder=get_embedder("local:BAAI/bge-small-en-v1.5"),
)
```

`push()` writes content to the volume first, then embeds and upserts content-free vectors — `(id, vector, {file_id, chunkset_index})`, in a single `poma_chunksets` table with the `vector` extension — via the connector. This is a simpler alternative to the hand-rolled two-table schema above when you don't need the relational-lineage JOIN; use whichever fits your retrieval pattern.

## Feed pgvector from the parser you already run

Pair this page with your ingestion side — the same chunksets, straight from your OCR/parsing tool's raw output:

- [Mistral OCR → pgvector](/pipelines/mistral-ocr-to-pgvector)
- [LlamaParse → pgvector](/pipelines/llamaparse-to-pgvector)
- [Azure Document Intelligence → pgvector](/pipelines/azure-document-intelligence-to-pgvector)
- [Unstructured.io → pgvector](/pipelines/unstructured-to-pgvector)
- [Docling → pgvector](/pipelines/docling-to-pgvector)
- [Marker → pgvector](/pipelines/marker-to-pgvector)
- [AWS Textract → pgvector](/pipelines/textract-to-pgvector)

Running a different store? The same chunk design applies: [Qdrant](/optimal-chunks-qdrant) · [Pinecone](/optimal-chunks-pinecone) · [Weaviate](/optimal-chunks-weaviate) · [Milvus](/optimal-chunks-milvus) · [Chroma](/optimal-chunks-chroma).

Fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [RAG architecture guide](/guides/rag-architecture/) · [POMA chunksets](/learn/chunking/chunksets).