Skip to content

Mistral OCR to pgvector: Chunking for RAG ​

The short answer: Mistral OCR gives you excellent per-page markdown; Postgres with pgvector gives you ANN search inside a relational database — the only store on this list where a document's hierarchy can live as actual foreign keys. Wired together naively — flatten, split, embed, one flat table — you get a worse Pinecone, not a better Postgres. The missing link is POMA: PrimeCut().ingest() consumes the raw /v1/ocr JSON (auto-detected), emits hierarchy-preserving chunksets, and those land as two joined tables whose lineage a single ANN query can JOIN back together.

What each end of the pipeline actually provides ​

Mistral OCR (/v1/ocr, mistral-ocr-latest) returns pages[], each with an index and a markdown string — headings, lists, and tables recovered within each page, image bytes inline when you set include_image_base64. What it doesn't provide: cross-page hierarchy (page 41's ## Termination clauses has no link to page 3's # Master Services Agreement) or retrieval units. Details: The Optimal Chunker for Mistral OCR.

pgvector provides a vector column type, HNSW and IVFFlat indexes, and cosine/L2/inner-product operators — inside a database that already has transactions, foreign keys, JOINs, and tsvector full-text search. What it doesn't provide: any opinion about your schema. It will index a flat table of fragments just as happily as a real hierarchy. Details: The Optimal Chunks for the Best Retrieval in pgvector.

The naive wiring, and where it breaks ​

The common recipe — "\n".join(p["markdown"] for p in pages) → RecursiveCharacterTextSplitter → embed → INSERT INTO items (text, embedding) — breaks in a pair-specific way:

  • One flat table wastes the reason you chose Postgres. Chunks that belong to the same section have no relation to JOIN, so retrieving context means application-side re-fetching — in the one database where lineage could have been a foreign key.
  • Mistral's page indices vanish at the join, so the page column holds nothing truthful and page-cited answers are gone.
  • Nothing is transactional about the pipeline. Split-then-insert loops half-commit on failure; a document's chunks, chunksets, and memberships should land atomically or not at all — which requires them to be modeled as related tables in the first place.

Chunksets fix this at the unit level: self-explanatory root-to-leaf paths whose members are rows you can JOIN. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss (methodology).

The pipeline, end to end ​

bash
pip install mistralai poma "psycopg[binary]" pgvector sentence-transformers
sql
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
    file_id     text NOT NULL,
    chunk_index int  NOT NULL,
    page        int,
    depth       int,
    content     text NOT NULL,
    PRIMARY KEY (file_id, chunk_index)
);

CREATE TABLE chunksets (
    id        bigserial PRIMARY KEY,
    file_id   text NOT NULL,
    to_embed  text NOT NULL,
    embedding vector(384) NOT NULL
);

CREATE TABLE chunkset_members (
    chunkset_id bigint NOT NULL REFERENCES chunksets(id),
    file_id     text   NOT NULL,
    chunk_index int    NOT NULL,
    FOREIGN KEY (file_id, chunk_index) REFERENCES chunks(file_id, chunk_index)
);

CREATE INDEX ON chunksets USING hnsw (embedding vector_cosine_ops);
python
import os
import psycopg
from pgvector.psycopg import register_vector
from mistralai import Mistral
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer

# 1. Mistral OCR — your existing call, unchanged.
mistral = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
ocr = mistral.ocr.process(
    model="mistral-ocr-latest",
    document={"type": "document_url", "document_url": "https://example.com/contract.pdf"},
    include_image_base64=True,
)
with open("contract.mistral-ocr.json", "w") as f:
    f.write(ocr.model_dump_json())

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.mistral-ocr.json",  # Mistral shape auto-detected
    download_dir="store",
    filename="contract.poma",  # keep the archive: its chunk records carry `page`
)
raw = unpack_poma_archive(poma_archive_path="store/contract.poma")

# 3. Transactional ingest: chunks, chunksets, memberships land atomically.
model = SentenceTransformer("all-MiniLM-L6-v2")
conn = psycopg.connect(os.environ["DATABASE_URL"])
register_vector(conn)
file_id = result.chunks[0].file_id

with conn.transaction():
    for c in raw["chunks"]:
        conn.execute(
            "INSERT INTO chunks (file_id, chunk_index, page, depth, content)"
            " VALUES (%s, %s, %s, %s, %s)",
            (file_id, c["chunk_index"], c.get("page"), c["depth"], c["content"]),
        )
    for cs in result.chunksets:
        (cs_id,) = conn.execute(
            "INSERT INTO chunksets (file_id, to_embed, embedding)"
            " VALUES (%s, %s, %s) RETURNING id",
            (file_id, cs.to_embed, model.encode(cs.to_embed)),
        ).fetchone()
        for chunk_index in cs.chunks:  # cs.chunks is a list of chunk indices
            conn.execute(
                "INSERT INTO chunkset_members (chunkset_id, file_id, chunk_index)"
                " VALUES (%s, %s, %s)",
                (cs_id, file_id, chunk_index),
            )

# 4. ANN + lineage JOIN in one round trip.
qvec = model.encode("What are the early termination conditions?")
rows = conn.execute(
    """
    SELECT cs.id, c.depth, c.page, c.content
    FROM chunksets cs
    JOIN chunkset_members m ON m.chunkset_id = cs.id
    JOIN chunks c ON (c.file_id, c.chunk_index) = (m.file_id, m.chunk_index)
    WHERE cs.id IN (
        SELECT id FROM chunksets
        WHERE file_id = %s
        ORDER BY embedding <=> %s
        LIMIT 3
    )
    ORDER BY cs.id, c.depth, c.chunk_index
    """,
    (file_id, qvec),
).fetchall()
cheatsheet = "\n\n".join(content for _, _, _, content in rows)

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles all three Mistral image cases (base64 → described; annotation → spliced; neither → visible marker, counted in content_metadata), strips running headers/footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "mistral" or "none". For hybrid retrieval, add a generated tsvector column over to_embed with a GIN index and fuse lexical and vector rankings.

Metadata mapping: POMA fields → pgvector/Postgres primitives ​

POMA chunk fieldPostgres primitiveWhat it enables
to_embedvector(384) column, HNSW-indexed (<=>)paraphrase ANN retrieval
to_embed texttsvector generated column + GIN indexexact-term hybrid retrieval
file_idtext column in a WHERE clausescope the ANN subquery to one document
page (from Mistral pages[].index)int columnpage-cited answers, page-range predicates
depthint columnlineage ordering, depth filters
chunk_indexint column, part of the primary keystable ordering, foreign-key identity
chunkset lineagechunkset_members join table with foreign keysfull lineage via JOIN in the ANN query

Frequently asked questions ​

How do I get Mistral OCR results into Postgres with pgvector for RAG? ​

Save the raw /v1/ocr JSON, run PrimeCut().ingest() on it (auto-detected, hierarchy rebuilt), store chunks and chunksets as joined tables, embed to_embed into a vector column, and query with ORDER BY embedding <=> %s plus a lineage JOIN.

Should Mistral OCR chunks go into one flat table in pgvector? ​

No — a flat table of split text turns Postgres into a bare vector store. Joined chunk and chunkset tables with foreign keys make the hierarchy queryable SQL, which is the reason to pick pgvector at all.

How do I retrieve a chunkset's full lineage in one pgvector query? ​

ANN in a subquery (ORDER BY embedding <=> %s LIMIT k), then JOIN through chunkset_members to chunks, ordered by depth and chunk_index. One round trip, full lineage.

Should I use HNSW or IVFFlat for Mistral OCR chunksets in pgvector? ​

HNSW for most document pipelines — it works on an empty table and takes incremental inserts. IVFFlat builds faster but clusters existing rows, so it wants representative data first.

Can I combine pgvector similarity with Postgres full-text search for OCR'd documents? ​

Yes: a generated tsvector column with a GIN index next to the vector column, rankings fused at query time. It pays off because chunksets keep exact tokens like clause numbers intact.

Same parser, different store: Mistral OCR → Qdrant · Mistral OCR → Pinecone · Mistral OCR → Weaviate

Same store, different parser: Docling → pgvector · Marker → pgvector · Textract → pgvector

Foundations: The Optimal Chunker for Mistral OCR · The Optimal Chunks for pgvector · All pipeline recipes · RAG chunking guide