Skip to content

Docling to Pinecone: Chunking for RAG ​

The short answer: Docling hands you a typed DoclingDocument tree — explicit heading levels, cell-grid tables, page furniture already isolated. Pinecone hands you serverless vector search with namespaces, metadata filtering, and sparse-dense hybrid support. The naive wiring — export_to_markdown(), split, embed, upsert — wastes the first and misuses the second: structure is flattened away, and the usual "fix" of stuffing full text and heading paths into metadata collides with Pinecone's ~40 KB per-vector cap. The missing link is POMA: PrimeCut().ingest() parses the tree natively into chunksets, you embed each chunkset's to_embed text, and Pinecone stores compact {file_id, page, depth} metadata while the full text stays in the portable .poma archive.

What each end of the pipeline actually provides ​

Docling (IBM's open-source converter, also served via docling-serve) returns the DoclingDocument: a typed texts/tables/pictures/groups tree where section_header items carry an explicit numeric level, tables are cell grids, and repeating page furniture sits pre-classified in the furniture group with page_header/page_footer labels. What it doesn't decide: what an embedding vector should represent. Details: The Optimal Chunker for Docling.

Pinecone provides serverless indexes, namespaces as hard partitions (a natural fit for one namespace per tenant or corpus), metadata filtering with $eq/$in/$gte operators, and sparse-dense hybrid support. What it doesn't provide: any repair of what you embed. It also enforces a real constraint the naive pipeline trips over — a metadata budget of roughly 40 KB per vector. Details: The Optimal Chunks for the Best Retrieval in Pinecone.

The naive wiring, and where it breaks ​

The common recipe — doc.export_to_markdown() → RecursiveCharacterTextSplitter → embed → index.upsert(...) — breaks in a pair-specific way, in two stages:

  1. The flattening stage discards Docling's structure. Explicit section_header levels are reduced to # glyphs the splitter ignores; the furniture group's page headers, footers, and page numbers pour back into the text stream and become per-page noise vectors; overlap embeds every boundary span twice, inflating serverless storage with near-duplicates.
  2. The repair attempt hits the metadata cap. Teams notice the retrieved fragments arrive context-free, so they stuff the fix into Pinecone metadata: the full chunk text, the reconstructed heading path, sometimes a serialized table for good measure. On a deeply nested Docling section — a level-4 clause under three ancestor headings, next to an HTML table — that payload runs straight into Pinecone's ~40 KB per-vector metadata limit. Now the pipeline truncates the very context it was trying to preserve, silently, on exactly the documents where hierarchy matters most.

The fix is architectural, not cosmetic: make the retrieval unit self-explanatory before it reaches Pinecone, and store coordinates rather than content in metadata. That is what chunksets are — root-to-leaf units whose to_embed text already carries the ancestor breadcrumbs — so the metadata can stay three scalars small.

The pipeline, end to end ​

bash
pip install docling poma pinecone sentence-transformers
python
import json
import os
from docling.document_converter import DocumentConverter
from pinecone import Pinecone, ServerlessSpec
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer

# 1. Docling conversion — your existing call, unchanged.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

# 2. The missing link — DoclingDocument tree in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.docling.json",  # DoclingDocument shape auto-detected
    download_dir="store",
    filename="contract.poma",  # keep the archive — page numbers live in its chunk records
)

# A chunkset's structural metadata comes from its leaf chunk (cs.chunks is in
# document order, so the last index is the leaf). depth is on the SDK objects;
# page only exists on the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
    c["chunk_index"]: c.get("page")
    for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}

# 3. Embed chunksets; upsert with compact metadata (well under the 40 KB cap).
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
if not pc.has_index("contracts"):
    pc.create_index(
        name="contracts",
        dimension=384,
        metric="cosine",
        spec=ServerlessSpec(cloud="aws", region="us-east-1"),
    )
index = pc.Index("contracts")

index.upsert(
    vectors=[
        {
            "id": f"{cs.file_id}:{cs.chunkset_index}",
            "values": model.encode(cs.to_embed).tolist(),
            "metadata": {
                "file_id": cs.file_id,
                "chunkset_index": cs.chunkset_index,
                "page": page_of[cs.chunks[-1]],
                "depth": depth_of[cs.chunks[-1]],
            },
        }
        for cs in result.chunksets
    ],
    namespace="legal",  # one namespace per tenant/corpus
)

# 4. Query the namespace, join IDs back to full chunkset text.
query = "What are the early termination conditions?"
hits = index.query(
    vector=model.encode(query).tolist(),
    top_k=5,
    namespace="legal",
    include_metadata=True,
)
retrieved = [result.chunksets[int(m["id"].split(":", 1)[1])] for m in hits["matches"]]
context = "\n\n".join(dict.fromkeys(cs.to_embed for cs in retrieved))  # dedupe = cheatsheet step

POMA validates the payload up front — auto-detection fingerprints schema_name == "DoclingDocument", and a corrupted or mislabeled upload 422s immediately rather than silently degrading the index. A docling-serve envelope carrying only md_content falls back to a markdown passthrough. To force or suppress detection, set external_ocr_source to "docling" or "none". In production, the ID-to-text join runs against the portable .poma archive or your document store instead of an in-memory result, and the final dedupe-and-merge is the cheatsheet assembly step.

The payoff is measurable: on our reference legal-document benchmark, chunksets plus cheatsheet assembly answered the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss — methodology in Document Ingestion & Chunking for RAG.

Metadata mapping: POMA fields → Pinecone primitives ​

POMA chunk fieldPinecone primitiveWhat it enables
to_embeddense vector values (+ optional sparse values for hybrid)paraphrase + exact-term retrieval
file_idmetadata field, filter with $eq/$inscope queries to one document
page (on the archive's chunk records, from Docling item provenance; take the chunkset's leaf chunk)metadata field, filter with $gte/$ltepage-cited answers, page-range filters
depth of the leaf chunk (from explicit section_header levels)metadata fieldfilter/re-rank by hierarchy level
chunkset_indexencoded in the vector ID ({file_id}:{chunkset_index})stable ordering, ID-to-text join
chunkset full textnot metadata — .poma archive / doc storestays clear of the ~40 KB cap

Tenant or corpus boundaries map to namespaces; document boundaries map to the file_id filter within a namespace.

Frequently asked questions ​

How do I get Docling output into Pinecone for RAG? ​

Save export_to_dict() as JSON, run PrimeCut().ingest() on it (auto-detected, tree parsed natively), embed each chunkset's to_embed, and upsert with {file_id, chunkset_index, page, depth} metadata into a namespace — depth off the SDK objects, page from the .poma archive's chunk records for the chunkset's leaf chunk. Join returned IDs back to full text for the prompt.

How do I keep Docling chunk metadata under Pinecone's 40 KB limit? ​

Store coordinates, not content: a handful of compact scalars in metadata, full chunkset text in the .poma archive or your document store, vector IDs as the join key. Deep heading paths and HTML tables never touch the cap.

Should each Docling document get its own Pinecone namespace? ​

Namespaces for tenants or corpora — they are hard partitions queries never cross. Documents are scoped inside a namespace with a file_id $eq filter, keeping cross-document search available.

Does sparse-dense hybrid search in Pinecone help with Docling-parsed documents? ​

Yes — clause numbers, part codes, and defined terms are exact tokens dense embeddings blur. Attach a sparse representation alongside each dense vector; chunkset to_embed text includes heading breadcrumbs, so the sparse side matches section titles too.

Why not export Docling to markdown and split it before upserting to Pinecone? ​

Flattening collapses explicit heading levels to glyphs, re-injects the pre-isolated furniture noise, and overlap multiplies near-duplicate vectors. The usual metadata "fix" then collides with the 40 KB cap. Parse the tree; embed chunksets.

Same parser, different store: Docling → Qdrant · Docling → Weaviate · Docling → Milvus

Same store, different parser: Mistral OCR → Pinecone · Unstructured → Pinecone · Textract → Pinecone

Foundations: The Optimal Chunker for Docling · The Optimal Chunks for Pinecone · All pipeline recipes · RAG chunking guide