Docling to Weaviate: Chunking for RAG
The short answer: Docling gives you a typed DoclingDocument tree — explicit heading levels, cell-grid tables, page furniture already isolated. Weaviate gives you built-in hybrid search: BM25F keyword scoring and vector similarity fused by an alpha blend, over filterable properties. Wired naively — export_to_markdown(), split, embed — the pairing sabotages itself: flattening re-injects the furniture Docling had classified away, and BM25F dutifully scores every repeated running title. The missing link is POMA: PrimeCut().ingest() parses the tree natively into chunksets, which you insert into a Weaviate collection with file_id/page/depth properties and query with collection.query.hybrid — both halves of the hybrid operating on clean, self-explanatory units.
What each end of the pipeline actually provides
Docling (IBM's open-source converter, also served via docling-serve) returns the DoclingDocument: a typed texts/tables/pictures/groups tree where section_header items carry an explicit numeric level, tables are cell grids, and repeating page furniture sits pre-classified in the furniture group with page_header/page_footer labels. What it doesn't decide: what a retrieval unit should be. Details: The Optimal Chunker for Docling.
Weaviate provides collections with named vectors, hybrid search that fuses BM25F and vector scores under a tunable alpha, filterable properties for scoped queries, and cross-references between collections — enough to model chunk-to-chunkset relations inside the database. What it doesn't provide: any repair of what you store. BM25F in particular scores whatever tokens you give it, noise included. Details: The Optimal Chunks for the Best Retrieval in Weaviate.
The naive wiring, and where it breaks
The common recipe — doc.export_to_markdown() → RecursiveCharacterTextSplitter → embed → insert — breaks in a pair-specific way, and Weaviate's keyword half takes the worst of it:
- Furniture becomes BM25F term spam. Docling had parked running headers, footers, and page numbers in the
furnituregroup. Flattening pours them back into the text stream, so a running title like Confidential — Master Services Agreement appears in the stored text of chunks from every page. BM25F now sees those tokens everywhere: keyword scores skew toward whichever fragments happen to contain more furniture, and queries mentioning the document title match half the collection indiscriminately. Noalphasetting fixes a polluted keyword index — tuning the blend just decides how much of the noise you weight. - Explicit heading levels collapse. A
section_headerwithlevel: 2becomes a##glyph the splitter ignores; the fragment under Early termination arrives with no lineage, so even a well-ranked hit reads out of context. - Overlap double-counts terms. Splitter overlap stores every boundary span twice — near-duplicate objects that crowd top-k and double-count their tokens in BM25F document statistics.
POMA parses the tree natively instead: the furniture classification is honored and the noise dropped before insertion (deterministic, and free — your precursor did the work), explicit levels become depth, and overlap-free chunksets carry their ancestor breadcrumbs in to_embed — so BM25F matches section titles because they belong to the unit, not because flattening smeared them everywhere.
The pipeline, end to end
pip install docling poma weaviate-client sentence-transformersimport json
import os
import weaviate
from docling.document_converter import DocumentConverter
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.query import Filter
# 1. Docling conversion — your existing call, unchanged.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
json.dump(doc.export_to_dict(), f)
# 2. The missing link — DoclingDocument tree in, chunks + chunksets out.
poma = PrimeCut() # reads POMA_API_KEY
result = poma.ingest(
"contract.docling.json", # DoclingDocument shape auto-detected
download_dir="store",
filename="contract.poma", # keep the archive — page numbers live in its chunk records
)
# A chunkset's structural metadata comes from its leaf chunk (cs.chunks is in
# document order, so the last index is the leaf). depth is on the SDK objects;
# page only exists on the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
c["chunk_index"]: c.get("page")
for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}
# 3. Weaviate — one chunkset collection, hierarchy as filterable properties.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
client = weaviate.connect_to_local() # or connect_to_weaviate_cloud(...)
if not client.collections.exists("Chunkset"):
client.collections.create(
"Chunkset",
vectorizer_config=Configure.Vectorizer.none(), # we bring our own vectors
properties=[
Property(name="content", data_type=DataType.TEXT),
Property(name="file_id", data_type=DataType.TEXT),
Property(name="chunkset_index", data_type=DataType.INT),
Property(name="page", data_type=DataType.INT),
Property(name="depth", data_type=DataType.INT),
],
)
chunksets = client.collections.get("Chunkset")
with chunksets.batch.dynamic() as batch:
for cs in result.chunksets:
batch.add_object(
properties={
"content": cs.to_embed, # breadcrumbs included — BM25F sees them too
"file_id": cs.file_id,
"chunkset_index": cs.chunkset_index,
"page": page_of[cs.chunks[-1]],
"depth": depth_of[cs.chunks[-1]],
},
vector=model.encode(cs.to_embed).tolist(),
)
# 4. Hybrid retrieval — BM25F + vector, fused by alpha.
query = "What are the early termination conditions?"
hits = chunksets.query.hybrid(
query=query,
vector=model.encode(query).tolist(),
alpha=0.5, # 0 = pure BM25F, 1 = pure vector
limit=5,
filters=Filter.by_property("file_id").equal(result.chunksets[0].file_id),
)
context = "\n\n".join(dict.fromkeys(o.properties["content"] for o in hits.objects))
client.close()POMA validates the payload up front — auto-detection fingerprints schema_name == "DoclingDocument", and a corrupted or mislabeled upload 422s immediately rather than silently degrading the index. A docling-serve envelope carrying only md_content falls back to a markdown passthrough. To force or suppress detection, set external_ocr_source to "docling" or "none". The final dedupe-and-merge line is the cheatsheet assembly step; on our reference legal-document benchmark it delivers the same answer from 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss — methodology in Document Ingestion & Chunking for RAG.
Want chunk-level granularity as well? Weaviate's cross-references model the relation natively: a Chunk collection referencing its parent Chunkset object, so a fine-grained hit can be followed to its full root-to-leaf context in one traversal — no second store required.
Metadata mapping: POMA fields → Weaviate primitives
| POMA chunk field | Weaviate primitive | What it enables |
|---|---|---|
to_embed | content TEXT property + self-provided vector | BM25F and vector halves of hybrid, same unit |
file_id | TEXT property, filterable | scope queries to one document |
page (on the archive's chunk records, from Docling item provenance; take the chunkset's leaf chunk) | INT property, filterable | page-cited answers, page-range filters |
depth of the leaf chunk (from explicit section_header levels) | INT property, filterable | filter/re-rank by hierarchy level |
chunkset_index | INT property | stable ordering at assembly time |
| chunk ↔ chunkset lineage | cross-reference between collections | traverse a hit to its full context in-database |
Frequently asked questions
How do I get Docling output into Weaviate for RAG?
Save export_to_dict() as JSON, run PrimeCut().ingest() on it (auto-detected, tree parsed natively), insert chunksets into a collection with file_id/chunkset_index/page/depth properties and self-provided vectors — depth off the SDK objects, page from the .poma archive's chunk records for the chunkset's leaf chunk, then retrieve with collection.query.hybrid and assemble the cheatsheet.
What alpha should I use for Weaviate hybrid search over Docling-parsed documents?
Start at 0.5 and tune against your queries: lower weights BM25F for exact-term-heavy corpora, higher weights vectors for conversational questions. Chunksets serve both halves, since to_embed carries heading breadcrumbs alongside body text.
Do Docling's page headers and footers pollute Weaviate's BM25F scoring?
Only in the flatten-first pipeline, where repeated running titles become high-frequency term spam. POMA honors Docling's furniture classification and drops the noise before insertion, with its own header/footer strip on top.
How should I model chunks and chunksets in Weaviate collections?
One chunkset collection with filterable file_id/page/depth is enough for most stacks. For chunk-level granularity, add a Chunk collection cross-referencing its parent Chunkset — lineage traversal stays in-database.
Which Weaviate properties should be filterable for Docling content?
file_id (TEXT) for document scoping, page (INT) for page-cited answers, depth (INT) for hierarchy-aware filtering — with depth taken from Docling's explicit section_header levels, not re-inferred glyphs.
Related recipes
Same parser, different store: Docling → Qdrant · Docling → Pinecone · Docling → Chroma
Same store, different parser: Mistral OCR → Weaviate · LlamaParse → Weaviate · Azure Document Intelligence → Weaviate
Foundations: The Optimal Chunker for Docling · The Optimal Chunks for Weaviate · All pipeline recipes · RAG chunking guide