Marker to pgvector: Chunking for RAG
The short answer: Marker turns PDFs into clean structure on your own hardware; pgvector adds ANN search to the Postgres you already operate. Wired together naively — flatten the markdown, split, embed, INSERT into one flat table — the pairing wastes both: page identity is gone, and Postgres's defining strength, the JOIN, has nothing to join. The missing link is POMA: PrimeCut().ingest() consumes the saved Marker JSON (the Document block tree auto-routes), emits hierarchy-preserving chunksets, and those land as two joined tables — so one SQL statement runs the ANN search and retrieves each hit's full root-to-leaf lineage.
What each end of the pipeline actually provides
Marker (by Datalab / VikParuchuri) is the fast open-source PDF→markdown favourite for local pipelines. With the JSON renderer (--output_format json) it returns a Document block tree: a root block_type: "Document" with Page children, per-block HTML content, tables as full <table> elements with row and column spans, and a side-channel images dict of {name: base64}. What it doesn't provide: cross-page hierarchy or retrieval units — its job ends at "here is markdown." Details: The Optimal Chunker for Marker.
pgvector adds a vector column type, HNSW and IVFFlat indexes, and cosine/L2/inner-product operators to Postgres — alongside everything Postgres already does: transactions, B-tree indexes, tsvector full-text search, and JOINs. What it doesn't provide: any opinion about what a row should contain. It finds nearest neighbors of whatever you embedded — including context-free fragments. Details: The Optimal Chunks for the Best Retrieval in pgvector.
The naive wiring, and where it breaks
The common recipe — rendered.markdown → RecursiveCharacterTextSplitter → embed → INSERT INTO rag_fragments (text, embedding) — breaks in a pair-specific way:
- The
pagecolumn is NULL on every row. Marker's markdown output is one flat string with no page boundaries, so the schema's most useful filter column can never be populated. Page-cited answers are off the table before the first query. - One flat table means nothing to JOIN. Splitter fragments have no identity — no
file_id/chunk_indexkey, no membership in anything larger — so the relational engine you chose pgvector for is reduced to a key-value store with a vector column. - Overlap bloats the HNSW graph. Splitter overlap embeds every boundary span twice; in pgvector that means a larger index and top-k results where rows 2 and 3 are near-duplicates of row 1.
- Non-transactional trickle ingest. A script that embeds and inserts row by row, then dies mid-document, leaves a half-indexed document that silently answers queries with partial context.
- Dangling
refs from Marker's side-channel images dict end up embedded andtsvector-indexed as noise tokens.
The schema: chunks and chunksets as joined tables
POMA's output is naturally relational — chunks are rows, chunksets are groups of rows — which is why pgvector is the one store where the model maps to the engine's native primitive:
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (
file_id TEXT NOT NULL,
chunk_index INTEGER NOT NULL,
page INTEGER,
depth INTEGER,
content TEXT NOT NULL,
PRIMARY KEY (file_id, chunk_index)
);
CREATE TABLE chunksets (
id BIGSERIAL PRIMARY KEY,
file_id TEXT NOT NULL,
chunk_indices INTEGER[] NOT NULL,
to_embed TEXT NOT NULL,
embedding vector(384) NOT NULL,
tsv tsvector GENERATED ALWAYS AS (to_tsvector('english', to_embed)) STORED
);
CREATE INDEX ON chunksets USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON chunksets USING gin (tsv);
CREATE INDEX ON chunks (file_id, page);The pipeline, end to end
pip install marker-pdf poma "psycopg[binary]" pgvector sentence-transformers
marker_single contract.pdf --output_format json --output_dir out/import os
import psycopg
from pgvector.psycopg import register_vector
from sentence_transformers import SentenceTransformer
from poma import PrimeCut
from poma.utils import unpack_poma_archive
# 1. The missing link — raw Marker JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"out/contract/contract.json", # Document block tree auto-detected
download_dir="store",
filename="contract.poma", # keep the archive: its chunk records carry `page`
)
raw = unpack_poma_archive(poma_archive_path="store/contract.poma")
# 2. Embed each chunkset's normalized text.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
embeddings = model.encode([cs.to_embed for cs in result.chunksets])
# 3. Transactional ingest — the document lands atomically or not at all.
with psycopg.connect(os.environ["DATABASE_URL"]) as conn:
register_vector(conn)
with conn.transaction():
for c in raw["chunks"]:
conn.execute(
"INSERT INTO chunks (file_id, chunk_index, page, depth, content)"
" VALUES (%s, %s, %s, %s, %s)",
(c["file_id"], c["chunk_index"], c.get("page"), c["depth"], c["content"]),
)
for cs, emb in zip(result.chunksets, embeddings):
conn.execute(
"INSERT INTO chunksets (file_id, chunk_indices, to_embed, embedding)"
" VALUES (%s, %s, %s, %s)",
(cs.file_id, cs.chunks, cs.to_embed, emb), # cs.chunks is a list of chunk indices
)Retrieval is where the JOIN pays off — ANN search, full-text rank, and lineage in one statement:
query = "What are the early termination conditions?"
qvec = model.encode(query)
rows = conn.execute(
"""
WITH hits AS (
SELECT id, file_id, chunk_indices,
1 - (embedding <=> %(v)s) AS dense_score,
ts_rank(tsv, plainto_tsquery('english', %(q)s)) AS text_score
FROM chunksets
ORDER BY embedding <=> %(v)s
LIMIT 5
)
SELECT h.dense_score + h.text_score AS score,
c.chunk_index, c.page, c.depth, c.content
FROM hits h
JOIN chunks c
ON c.file_id = h.file_id
AND c.chunk_index = ANY (h.chunk_indices)
ORDER BY score DESC, c.chunk_index
""",
{"v": qvec, "q": query},
).fetchall()Deduplicate and merge the retrieved rows into a cheatsheet — one prompt-ready context block — before handing them to the LLM. POMA validates the upload shape up front (a corrupted or mislabeled payload 422s immediately), splices Marker's side-channel image bytes into described text, strips running headers and footers, and rebuilds the cross-page heading tree before chunking. Bare markdown shapes ({markdown, images, metadata}) are not auto-routed — declare external_ocr_source="marker", or "none" to opt out of detection.
The payoff, measured on our reference legal-document benchmark: 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, zero information loss. Methodology: Document Ingestion & Chunking for RAG.
Metadata mapping: POMA fields → Postgres primitives
| POMA chunk field | Postgres primitive | What it enables |
|---|---|---|
to_embed | vector(384) + HNSW index; generated tsvector + GIN index | dense + full-text hybrid in one statement |
file_id | TEXT, part of the primary key | per-document scoping, the lineage JOIN key |
page (from Marker's Page blocks) | INTEGER, B-tree indexed | page-cited answers, page-range WHERE |
depth | INTEGER | filter or re-rank by hierarchy level |
chunk_index | INTEGER, part of the primary key | stable ordering at assembly time |
| chunkset lineage | chunk_indices INTEGER[] + JOIN | full root-to-leaf context in the ANN query |
Frequently asked questions
How do I get Marker output into pgvector for RAG?
Run Marker with the JSON renderer, hand the saved result to PrimeCut().ingest() (auto-detected, hierarchy rebuilt), embed each chunkset's to_embed, and INSERT chunks and chunksets into two joined tables in one transaction. Retrieval is one SQL statement: ANN over chunkset embeddings, JOINed to chunks for lineage.
What Postgres schema should I use for Marker chunks with pgvector?
Two joined tables: chunks (file_id, chunk_index, page, depth, content) and chunksets with chunk_indices, to_embed, an HNSW-indexed vector column, and a GIN-indexed generated tsvector. One JOIN retrieves a chunkset's full root-to-leaf lineage alongside the ANN hit.
Can I combine pgvector vector search with Postgres full-text search on Marker output?
Yes, in one statement — rank by cosine distance and ts_rank together. Parsed documents are full of exact tokens (clause numbers, part codes, defined terms) that dense embeddings blur; the hybrid catches both, with no infrastructure beyond Postgres.
Why are page numbers missing when I load Marker output into Postgres?
Marker's default markdown output has no page boundaries, so the page column is NULL on every row. Use --output_format json: the Document block tree keeps per-page blocks, and POMA carries page onto every chunk.
Do I need new infrastructure to run RAG on Marker output?
No. Marker runs on your hardware; Postgres you already operate gets CREATE EXTENSION vector and two tables; ingest is a normal transaction. The only new piece is the chunking layer, and POMA's BYOCR connector fills it from the saved result — no re-parsing.
Related recipes
Same parser, different store: Marker → Qdrant · Marker → Milvus · Marker → Chroma
Same store, different parser: Mistral OCR → pgvector · Docling → pgvector · Unstructured → pgvector
Foundations: The Optimal Chunker for Marker · The Optimal Chunks for pgvector · All pipeline recipes · RAG chunking guide