Docling to Pinecone: Chunking for RAG
The short answer: Docling hands you a typed DoclingDocument tree — explicit heading levels, cell-grid tables, page furniture already isolated. Pinecone hands you serverless vector search with namespaces, metadata filtering, and sparse-dense hybrid support. The naive wiring — export_to_markdown(), split, embed, upsert — wastes the first and misuses the second: structure is flattened away, and the usual "fix" of stuffing full text and heading paths into metadata collides with Pinecone's ~40 KB per-vector cap. The missing link is POMA: PrimeCut().ingest() parses the tree natively into chunksets, you embed each chunkset's to_embed text, and Pinecone stores compact {file_id, page, depth} metadata while the full text stays in the portable .poma archive.
What each end of the pipeline actually provides
Docling (IBM's open-source converter, also served via docling-serve) returns the DoclingDocument: a typed texts/tables/pictures/groups tree where section_header items carry an explicit numeric level, tables are cell grids, and repeating page furniture sits pre-classified in the furniture group with page_header/page_footer labels. What it doesn't decide: what an embedding vector should represent. Details: The Optimal Chunker for Docling.
Pinecone provides serverless indexes, namespaces as hard partitions (a natural fit for one namespace per tenant or corpus), metadata filtering with $eq/$in/$gte operators, and sparse-dense hybrid support. What it doesn't provide: any repair of what you embed. It also enforces a real constraint the naive pipeline trips over — a metadata budget of roughly 40 KB per vector. Details: The Optimal Chunks for the Best Retrieval in Pinecone.
The naive wiring, and where it breaks
The common recipe — doc.export_to_markdown() → RecursiveCharacterTextSplitter → embed → index.upsert(...) — breaks in a pair-specific way, in two stages:
- The flattening stage discards Docling's structure. Explicit
section_headerlevels are reduced to#glyphs the splitter ignores; thefurnituregroup's page headers, footers, and page numbers pour back into the text stream and become per-page noise vectors; overlap embeds every boundary span twice, inflating serverless storage with near-duplicates. - The repair attempt hits the metadata cap. Teams notice the retrieved fragments arrive context-free, so they stuff the fix into Pinecone metadata: the full chunk text, the reconstructed heading path, sometimes a serialized table for good measure. On a deeply nested Docling section — a level-4 clause under three ancestor headings, next to an HTML table — that payload runs straight into Pinecone's ~40 KB per-vector metadata limit. Now the pipeline truncates the very context it was trying to preserve, silently, on exactly the documents where hierarchy matters most.
The fix is architectural, not cosmetic: make the retrieval unit self-explanatory before it reaches Pinecone, and store coordinates rather than content in metadata. That is what chunksets are — root-to-leaf units whose to_embed text already carries the ancestor breadcrumbs — so the metadata can stay three scalars small.
The pipeline, end to end
pip install docling poma pinecone sentence-transformersimport json
import os
from docling.document_converter import DocumentConverter
from pinecone import Pinecone, ServerlessSpec
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer
# 1. Docling conversion — your existing call, unchanged.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
json.dump(doc.export_to_dict(), f)
# 2. The missing link — DoclingDocument tree in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.docling.json", # DoclingDocument shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive — page numbers live in its chunk records
)
# A chunkset's structural metadata comes from its leaf chunk (cs.chunks is in
# document order, so the last index is the leaf). depth is on the SDK objects;
# page only exists on the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
c["chunk_index"]: c.get("page")
for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}
# 3. Embed chunksets; upsert with compact metadata (well under the 40 KB cap).
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
if not pc.has_index("contracts"):
pc.create_index(
name="contracts",
dimension=384,
metric="cosine",
spec=ServerlessSpec(cloud="aws", region="us-east-1"),
)
index = pc.Index("contracts")
index.upsert(
vectors=[
{
"id": f"{cs.file_id}:{cs.chunkset_index}",
"values": model.encode(cs.to_embed).tolist(),
"metadata": {
"file_id": cs.file_id,
"chunkset_index": cs.chunkset_index,
"page": page_of[cs.chunks[-1]],
"depth": depth_of[cs.chunks[-1]],
},
}
for cs in result.chunksets
],
namespace="legal", # one namespace per tenant/corpus
)
# 4. Query the namespace, join IDs back to full chunkset text.
query = "What are the early termination conditions?"
hits = index.query(
vector=model.encode(query).tolist(),
top_k=5,
namespace="legal",
include_metadata=True,
)
retrieved = [result.chunksets[int(m["id"].split(":", 1)[1])] for m in hits["matches"]]
context = "\n\n".join(dict.fromkeys(cs.to_embed for cs in retrieved)) # dedupe = cheatsheet stepPOMA validates the payload up front — auto-detection fingerprints schema_name == "DoclingDocument", and a corrupted or mislabeled upload 422s immediately rather than silently degrading the index. A docling-serve envelope carrying only md_content falls back to a markdown passthrough. To force or suppress detection, set external_ocr_source to "docling" or "none". In production, the ID-to-text join runs against the portable .poma archive or your document store instead of an in-memory result, and the final dedupe-and-merge is the cheatsheet assembly step.
The payoff is measurable: on our reference legal-document benchmark, chunksets plus cheatsheet assembly answered the same query with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss — methodology in Document Ingestion & Chunking for RAG.
Metadata mapping: POMA fields → Pinecone primitives
| POMA chunk field | Pinecone primitive | What it enables |
|---|---|---|
to_embed | dense vector values (+ optional sparse values for hybrid) | paraphrase + exact-term retrieval |
file_id | metadata field, filter with $eq/$in | scope queries to one document |
page (on the archive's chunk records, from Docling item provenance; take the chunkset's leaf chunk) | metadata field, filter with $gte/$lte | page-cited answers, page-range filters |
depth of the leaf chunk (from explicit section_header levels) | metadata field | filter/re-rank by hierarchy level |
chunkset_index | encoded in the vector ID ({file_id}:{chunkset_index}) | stable ordering, ID-to-text join |
| chunkset full text | not metadata — .poma archive / doc store | stays clear of the ~40 KB cap |
Tenant or corpus boundaries map to namespaces; document boundaries map to the file_id filter within a namespace.
Frequently asked questions
How do I get Docling output into Pinecone for RAG?
Save export_to_dict() as JSON, run PrimeCut().ingest() on it (auto-detected, tree parsed natively), embed each chunkset's to_embed, and upsert with {file_id, chunkset_index, page, depth} metadata into a namespace — depth off the SDK objects, page from the .poma archive's chunk records for the chunkset's leaf chunk. Join returned IDs back to full text for the prompt.
How do I keep Docling chunk metadata under Pinecone's 40 KB limit?
Store coordinates, not content: a handful of compact scalars in metadata, full chunkset text in the .poma archive or your document store, vector IDs as the join key. Deep heading paths and HTML tables never touch the cap.
Should each Docling document get its own Pinecone namespace?
Namespaces for tenants or corpora — they are hard partitions queries never cross. Documents are scoped inside a namespace with a file_id $eq filter, keeping cross-document search available.
Does sparse-dense hybrid search in Pinecone help with Docling-parsed documents?
Yes — clause numbers, part codes, and defined terms are exact tokens dense embeddings blur. Attach a sparse representation alongside each dense vector; chunkset to_embed text includes heading breadcrumbs, so the sparse side matches section titles too.
Why not export Docling to markdown and split it before upserting to Pinecone?
Flattening collapses explicit heading levels to glyphs, re-injects the pre-isolated furniture noise, and overlap multiplies near-duplicate vectors. The usual metadata "fix" then collides with the 40 KB cap. Parse the tree; embed chunksets.
Related recipes
Same parser, different store: Docling → Qdrant · Docling → Weaviate · Docling → Milvus
Same store, different parser: Mistral OCR → Pinecone · Unstructured → Pinecone · Textract → Pinecone
Foundations: The Optimal Chunker for Docling · The Optimal Chunks for Pinecone · All pipeline recipes · RAG chunking guide