Mistral OCR to pgvector: Chunking for RAG
The short answer: Mistral OCR gives you excellent per-page markdown; Postgres with pgvector gives you ANN search inside a relational database — the only store on this list where a document's hierarchy can live as actual foreign keys. Wired together naively — flatten, split, embed, one flat table — you get a worse Pinecone, not a better Postgres. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON (auto-detected), emits hierarchy-preserving chunksets, and those land as two joined tables whose lineage a single ANN query can JOIN back together.
What each end of the pipeline actually provides
Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, image bytes inline when you set include_image_base64. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for Mistral OCR.
pgvector provides a vector column type, HNSW and IVFFlat indexes, and cosine/L2/inner-product operators — inside a database that already has transactions, foreign keys, JOINs, and tsvector full-text search. What it doesn't provide: any opinion about your schema. It will index a flat table of fragments just as happily as a real hierarchy. Details: The Optimal Chunks for the Best Retrieval in pgvector.
The naive wiring, and where it breaks
The common recipe — "\n".join(p["markdown"] for p in pages) → RecursiveCharacterTextSplitter → embed → INSERT INTO items (text, embedding) — breaks in a pair-specific way:
- One flat table wastes the reason you chose Postgres. Chunks that belong to the same section have no relation to
JOIN, so retrieving context means application-side re-fetching — in the one database where lineage could have been a foreign key. - Mistral's page indices vanish at the join, so the
pagecolumn holds nothing truthful and page-cited answers are gone. - Nothing is transactional about the pipeline. Split-then-insert loops half-commit on failure; a document's chunks, chunksets, and memberships should land atomically or not at all — which requires them to be modeled as related tables in the first place.
Chunksets fix this at the unit level: self-explanatory root-to-leaf paths whose members are rows you can JOIN. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).
The pipeline, end to end
pip install mistralai poma "psycopg[binary]" pgvector sentence-transformersCREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (
file_id text NOT NULL,
chunk_index int NOT NULL,
page int,
depth int,
content text NOT NULL,
PRIMARY KEY (file_id, chunk_index)
);
CREATE TABLE chunksets (
id bigserial PRIMARY KEY,
file_id text NOT NULL,
to_embed text NOT NULL,
embedding vector(384) NOT NULL
);
CREATE TABLE chunkset_members (
chunkset_id bigint NOT NULL REFERENCES chunksets(id),
file_id text NOT NULL,
chunk_index int NOT NULL,
FOREIGN KEY (file_id, chunk_index) REFERENCES chunks(file_id, chunk_index)
);
CREATE INDEX ON chunksets USING hnsw (embedding vector_cosine_ops);import os
import psycopg
from pgvector.psycopg import register_vector
from mistralai import Mistral
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer
# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
model="mistral-ocr-latest",
document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
f.write(ocr.model_dump_json())
# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.mistral-ocr.json", # Mistral shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive: its chunk records carry `page`
)
raw = unpack_poma_archive(poma_archive_path="store/contract.poma")
# 3. Transactional ingest: chunks, chunksets, memberships land atomically.
model = SentenceTransformer("all-MiniLM-L6-v2")
conn = psycopg.connect(os.environ["DATABASE_URL"])
register_vector(conn)
file_id = result.chunks[0].file_id
with conn.transaction():
for c in raw["chunks"]:
conn.execute(
"INSERT INTO chunks (file_id, chunk_index, page, depth, content)"
" VALUES (%s, %s, %s, %s, %s)",
(file_id, c["chunk_index"], c.get("page"), c["depth"], c["content"]),
)
for cs in result.chunksets:
(cs_id,) = conn.execute(
"INSERT INTO chunksets (file_id, to_embed, embedding)"
" VALUES (%s, %s, %s) RETURNING id",
(file_id, cs.to_embed, model.encode(cs.to_embed)),
).fetchone()
for chunk_index in cs.chunks: # cs.chunks is a list of chunk indices
conn.execute(
"INSERT INTO chunkset_members (chunkset_id, file_id, chunk_index)"
" VALUES (%s, %s, %s)",
(cs_id, file_id, chunk_index),
)
# 4. ANN + lineage JOIN in one round trip.
qvec = model.encode("What are the early termination conditions?")
rows = conn.execute(
"""
SELECT cs.id, c.depth, c.page, c.content
FROM chunksets cs
JOIN chunkset_members m ON m.chunkset_id = cs.id
JOIN chunks c ON (c.file_id, c.chunk_index) = (m.file_id, m.chunk_index)
WHERE cs.id IN (
SELECT id FROM chunksets
WHERE file_id = %s
ORDER BY embedding <=> %s
LIMIT 3
)
ORDER BY cs.id, c.depth, c.chunk_index
""",
(file_id, qvec),
).fetchall()
cheatsheet = "\n\n".join(content for _, _, _, content in rows)POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases (base64 → described; annotation → spliced; neither → visible marker, counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "mistral" or "none". For hybrid retrieval, add a generated tsvector column over to_embed with a GIN index and fuse lexical and vector rankings.
Metadata mapping: POMA fields → pgvector/Postgres primitives
| POMA chunk field | Postgres primitive | What it enables |
|---|---|---|
to_embed | vector(384) column, HNSW-indexed (<=>) | paraphrase ANN retrieval |
to_embed text | tsvector generated column + GIN index | exact-term hybrid retrieval |
file_id | text column in a WHERE clause | scope the ANN subquery to one document |
page (from Mistral pages[].index) | int column | page-cited answers, page-range predicates |
depth | int column | lineage ordering, depth filters |
chunk_index | int column, part of the primary key | stable ordering, foreign-key identity |
| chunkset lineage | chunkset_members join table with foreign keys | full lineage via JOIN in the ANN query |
Frequently asked questions
How do I get Mistral OCR results into Postgres with pgvector for RAG?
Save the raw /v1/ocr JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), store chunks and chunksets as joined tables, embed to_embed into a vector column, and query with ORDER BY embedding <=> %s plus a lineage JOIN.
Should Mistral OCR chunks go into one flat table in pgvector?
No — a flat table of split text turns Postgres into a bare vector store. Joined chunk and chunkset tables with foreign keys make the hierarchy queryable SQL, which is the reason to pick pgvector at all.
How do I retrieve a chunkset's full lineage in one pgvector query?
ANN in a subquery (ORDER BY embedding <=> %s LIMIT k), then JOIN through chunkset_members to chunks, ordered by depth and chunk_index. One round trip, full lineage.
Should I use HNSW or IVFFlat for Mistral OCR chunksets in pgvector?
HNSW for most document pipelines — it works on an empty table and takes incremental inserts. IVFFlat builds faster but clusters existing rows, so it wants representative data first.
Can I combine pgvector similarity with Postgres full-text search for OCR'd documents?
Yes: a generated tsvector column with a GIN index next to the vector column, rankings fused at query time. It pays off because chunksets keep exact tokens like clause numbers intact.
Related recipes
Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Pinecone · Mistral OCR → Weaviate
Same store, different parser: Docling → pgvector · Marker → pgvector · Textract → pgvector
Foundations: The Optimal Chunker for Mistral OCR · The Optimal Chunks for pgvector · All pipeline recipes · RAG chunking guide