PaddleOCR-VL to pgvector: Chunking for RAG
The short answer: PaddleOCR-VL gives you self-hosted, layout-labeled document parsing; Postgres with pgvector gives you ANN search inside a relational database — the only store on this list where a document's hierarchy can live as actual foreign keys. Wired together naively — flatten the labeled blocks, split, embed, one flat table — you get a worse Pinecone running on your own hardware, not a better Postgres. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (auto-detected), emits hierarchy-preserving chunksets, and those land as two joined tables whose lineage a single ANN query can JOIN back together.
What each end of the pipeline actually provides
PaddleOCR-VL runs entirely on your own infrastructure — a layout-detection model (PP-DocLayoutV3) plus a vision-language OCR model served via vLLM behind a PaddleX-compatible /layout-parsing endpoint. It returns either a serving envelope with a pre-assembled markdown object, or the raw save_to_json() block list (parsing_res_list): labeled regions like doc_title, paragraph_title, table, and running furniture, each carrying a block_order that is the model's true reading order. What it doesn't provide: a link from a paragraph_title on page 12 back to the doc_title on page 1, or a decision about what a retrieval unit should be. Details: The Optimal Chunker for PaddleOCR-VL.
pgvector provides a vector column type, HNSW and IVFFlat indexes, and cosine/L2/inner-product operators — inside a database that already has transactions, foreign keys, JOINs, and tsvector full-text search. What it doesn't provide: any opinion about your schema. It will index a flat table of fragments just as happily as a real hierarchy. Details: The Optimal Chunks for the Best Retrieval in pgvector.
The naive wiring, and where it breaks
The common recipe — concatenate parsing_res_list block content (or markdown.text pages) → RecursiveCharacterTextSplitter → embed → INSERT INTO items (text, embedding) — breaks in a pair-specific way:
- Array position is not reading order. PaddleOCR-VL's
block_orderfield exists precisely because a multi-column layout can make the list index diverge from true reading order. A pipeline that iterates the raw list and assignschunk_indexfrom position, notblock_order, inserts rows whose sequence is wrong — and pgvector's lineageJOINruns clean anyway, returning the clauses in the wrong order with no error raised. - One flat table wastes the reason you chose Postgres. Labeled blocks that belong to the same section have no relation to
JOIN, so retrieving context means application-side re-fetching — in the one database where lineage could have been a foreign key. - Nothing is transactional about the pipeline. Split-then-insert loops half-commit on failure; a document's chunks, chunksets, and memberships should land atomically or not at all, which requires them to be modeled as related tables in the first place.
Chunksets fix this at the unit level: self-explanatory root-to-leaf paths whose members are rows you can JOIN, assembled from block_order, not array position. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install poma requests "psycopg[binary]" pgvector sentence-transformersCREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (
file_id text NOT NULL,
chunk_index int NOT NULL,
page int,
depth int,
content text NOT NULL,
PRIMARY KEY (file_id, chunk_index)
);
CREATE TABLE chunksets (
id bigserial PRIMARY KEY,
file_id text NOT NULL,
to_embed text NOT NULL,
embedding vector(384) NOT NULL
);
CREATE TABLE chunkset_members (
chunkset_id bigint NOT NULL REFERENCES chunksets(id),
file_id text NOT NULL,
chunk_index int NOT NULL,
FOREIGN KEY (file_id, chunk_index) REFERENCES chunks(file_id, chunk_index)
);
CREATE INDEX ON chunksets USING hnsw (embedding vector_cosine_ops);import json
import os
import requests
import psycopg
from pgvector.psycopg import register_vector
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.paddleocr-vl.json", # PaddleOCR-VL shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive: its chunk records carry `page`
)
raw = unpack_poma_archive(poma_archive_path="store/contract.poma")
# 3. Transactional ingest: chunks, chunksets, memberships land atomically.
model = SentenceTransformer("all-MiniLM-L6-v2")
conn = psycopg.connect(os.environ["DATABASE_URL"])
register_vector(conn)
file_id = result.chunks[0].file_id
with conn.transaction():
for c in raw["chunks"]:
conn.execute(
"INSERT INTO chunks (file_id, chunk_index, page, depth, content)"
" VALUES (%s, %s, %s, %s, %s)",
(file_id, c["chunk_index"], c.get("page"), c["depth"], c["content"]),
)
for cs in result.chunksets:
(cs_id,) = conn.execute(
"INSERT INTO chunksets (file_id, to_embed, embedding)"
" VALUES (%s, %s, %s) RETURNING id",
(file_id, cs.to_embed, model.encode(cs.to_embed)),
).fetchone()
for chunk_index in cs.chunks: # cs.chunks is a list of chunk indices
conn.execute(
"INSERT INTO chunkset_members (chunkset_id, file_id, chunk_index)"
" VALUES (%s, %s, %s)",
(cs_id, file_id, chunk_index),
)
# 4. ANN + lineage JOIN in one round trip.
qvec = model.encode("What are the early termination conditions?")
rows = conn.execute(
"""
SELECT cs.id, c.depth, c.page, c.content
FROM chunksets cs
JOIN chunkset_members m ON m.chunkset_id = cs.id
JOIN chunks c ON (c.file_id, c.chunk_index) = (m.file_id, m.chunk_index)
WHERE cs.id IN (
SELECT id FROM chunksets
WHERE file_id = %s
ORDER BY embedding <=> %s
LIMIT 3
)
ORDER BY cs.id, c.depth, c.chunk_index
""",
(file_id, qvec),
).fetchall()
cheatsheet = "\n\n".join(content for _, _, _, content in rows)POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles both PaddleOCR-VL shapes (serving envelope and raw save_to_json() block lists), assembles blocks by block_order rather than array position, drops running furniture (header/footer/page_number), and normalizes page_index (0-based, None only for a bare single image) to 1-based page numbers before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none".
Metadata mapping: POMA fields → pgvector/Postgres primitives
| POMA chunk field | Postgres primitive | What it enables |
|---|---|---|
to_embed | vector(384) column, HNSW-indexed (<=>) | paraphrase ANN retrieval |
to_embed text | tsvector generated column + GIN index | exact-term hybrid retrieval |
file_id | text column in a WHERE clause | scope the ANN subquery to one document |
page (from PaddleOCR-VL's 0-based page_index, normalized to 1-based) | int column | page-cited answers, page-range predicates |
depth | int column | lineage ordering, depth filters |
chunk_index (assigned by block_order, not array position) | int column, part of the primary key | stable, correctly-sequenced ordering |
| chunkset lineage | chunkset_members join table with foreign keys | full lineage via JOIN in the ANN query |
Frequently asked questions
How do I get PaddleOCR-VL results into Postgres with pgvector for RAG?
Save the raw /layout-parsing response, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt from block_order), store chunks and chunksets as joined tables, embed to_embed into a vector column, and query with ORDER BY embedding <=> %s plus a lineage JOIN.
Should PaddleOCR-VL blocks land in one flat pgvector table?
No — a flat table discards PaddleOCR-VL's own layout labels and turns Postgres into a bare vector store. Joined chunk and chunkset tables with foreign keys make the hierarchy queryable SQL, which is the reason to pick pgvector at all.
How do I retrieve a chunkset's full lineage in one pgvector query for PaddleOCR-VL content?
ANN in a subquery (ORDER BY embedding <=> %s LIMIT k), then JOIN through chunkset_members to chunks, ordered by depth and chunk_index. One round trip, full lineage, correctly sequenced.
Why does block order matter when storing self-hosted PaddleOCR-VL output in pgvector?
block_order is PaddleOCR-VL's true reading order; array position can diverge on multi-column layouts. Trusting position over block_order silently misorders chunk_index — the INSERT and the JOIN both succeed, but the returned lineage is wrong.
Can I combine pgvector similarity with full-text search for self-hosted PaddleOCR-VL documents?
Yes: a generated tsvector column with a GIN index next to the vector column, rankings fused at query time. It pays off because PaddleOCR-VL's layout labels already keep tables and headings separate from body text, so exact tokens stay intact inside each chunkset.
Related recipes
Same parser, different store: PaddleOCR-VL → Milvus · PaddleOCR-VL → Turbopuffer · PaddleOCR-VL → Vespa
Same store, different parser: Mistral OCR → pgvector · Docling → pgvector · Textract → pgvector
Foundations: The Optimal Chunker for PaddleOCR-VL · The Optimal Chunks for pgvector · All pipeline recipes · RAG chunking guide