Skip to content

Docling to Weaviate: Chunking for RAG ​

The short answer: Docling gives you a typed DoclingDocument tree — explicit heading levels, cell-grid tables, page furniture already isolated. Weaviate gives you built-in hybrid search: BM25F keyword scoring and vector similarity fused by an alpha blend, over filterable properties. Wired naively — export_to_markdown(), split, embed — the pairing sabotages itself: flattening re-injects the furniture Docling had classified away, and BM25F dutifully scores every repeated running title. The missing link is POMA: PrimeCut().ingest() parses the tree natively into chunksets, which you insert into a Weaviate collection with file_id/page/depth properties and query with collection.query.hybrid — both halves of the hybrid operating on clean, self-explanatory units.

What each end of the pipeline actually provides ​

Docling (IBM's open-source converter, also served via docling-serve) returns the DoclingDocument: a typed texts/tables/pictures/groups tree where section_header items carry an explicit numeric level, tables are cell grids, and repeating page furniture sits pre-classified in the furniture group with page_header/page_footer labels. What it doesn't decide: what a retrieval unit should be. Details: The Optimal Chunker for Docling.

Weaviate provides collections with named vectors, hybrid search that fuses BM25F and vector scores under a tunable alpha, filterable properties for scoped queries, and cross-references between collections — enough to model chunk-to-chunkset relations inside the database. What it doesn't provide: any repair of what you store. BM25F in particular scores whatever tokens you give it, noise included. Details: The Optimal Chunks for the Best Retrieval in Weaviate.

The naive wiring, and where it breaks ​

The common recipe — doc.export_to_markdown() → RecursiveCharacterTextSplitter → embed → insert — breaks in a pair-specific way, and Weaviate's keyword half takes the worst of it:

  • Furniture becomes BM25F term spam. Docling had parked running headers, footers, and page numbers in the furniture group. Flattening pours them back into the text stream, so a running title like Confidential — Master Services Agreement appears in the stored text of chunks from every page. BM25F now sees those tokens everywhere: keyword scores skew toward whichever fragments happen to contain more furniture, and queries mentioning the document title match half the collection indiscriminately. No alpha setting fixes a polluted keyword index — tuning the blend just decides how much of the noise you weight.
  • Explicit heading levels collapse. A section_header with level: 2 becomes a ## glyph the splitter ignores; the fragment under Early termination arrives with no lineage, so even a well-ranked hit reads out of context.
  • Overlap double-counts terms. Splitter overlap stores every boundary span twice — near-duplicate objects that crowd top-k and double-count their tokens in BM25F document statistics.

POMA parses the tree natively instead: the furniture classification is honored and the noise dropped before insertion (deterministic, and free — your precursor did the work), explicit levels become depth, and overlap-free chunksets carry their ancestor breadcrumbs in to_embed — so BM25F matches section titles because they belong to the unit, not because flattening smeared them everywhere.

The pipeline, end to end ​

bash
pip install docling poma weaviate-client sentence-transformers
python
import json
import os
import weaviate
from docling.document_converter import DocumentConverter
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.query import Filter

# 1. Docling conversion — your existing call, unchanged.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

# 2. The missing link — DoclingDocument tree in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.docling.json",  # DoclingDocument shape auto-detected
    download_dir="store",
    filename="contract.poma",  # keep the archive — page numbers live in its chunk records
)

# A chunkset's structural metadata comes from its leaf chunk (cs.chunks is in
# document order, so the last index is the leaf). depth is on the SDK objects;
# page only exists on the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
    c["chunk_index"]: c.get("page")
    for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}

# 3. Weaviate — one chunkset collection, hierarchy as filterable properties.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
client = weaviate.connect_to_local()  # or connect_to_weaviate_cloud(...)

if not client.collections.exists("Chunkset"):
    client.collections.create(
        "Chunkset",
        vectorizer_config=Configure.Vectorizer.none(),  # we bring our own vectors
        properties=[
            Property(name="content", data_type=DataType.TEXT),
            Property(name="file_id", data_type=DataType.TEXT),
            Property(name="chunkset_index", data_type=DataType.INT),
            Property(name="page", data_type=DataType.INT),
            Property(name="depth", data_type=DataType.INT),
        ],
    )
chunksets = client.collections.get("Chunkset")

with chunksets.batch.dynamic() as batch:
    for cs in result.chunksets:
        batch.add_object(
            properties={
                "content": cs.to_embed,  # breadcrumbs included — BM25F sees them too
                "file_id": cs.file_id,
                "chunkset_index": cs.chunkset_index,
                "page": page_of[cs.chunks[-1]],
                "depth": depth_of[cs.chunks[-1]],
            },
            vector=model.encode(cs.to_embed).tolist(),
        )

# 4. Hybrid retrieval — BM25F + vector, fused by alpha.
query = "What are the early termination conditions?"
hits = chunksets.query.hybrid(
    query=query,
    vector=model.encode(query).tolist(),
    alpha=0.5,  # 0 = pure BM25F, 1 = pure vector
    limit=5,
    filters=Filter.by_property("file_id").equal(result.chunksets[0].file_id),
)
context = "\n\n".join(dict.fromkeys(o.properties["content"] for o in hits.objects))
client.close()

POMA validates the payload up front — auto-detection fingerprints schema_name == "DoclingDocument", and a corrupted or mislabeled upload 422s immediately rather than silently degrading the index. A docling-serve envelope carrying only md_content falls back to a markdown passthrough. To force or suppress detection, set external_ocr_source to "docling" or "none". The final dedupe-and-merge line is the cheatsheet assembly step; on our reference legal-document benchmark it delivers the same answer from 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss — methodology in Document Ingestion & Chunking for RAG.

Want chunk-level granularity as well? Weaviate's cross-references model the relation natively: a Chunk collection referencing its parent Chunkset object, so a fine-grained hit can be followed to its full root-to-leaf context in one traversal — no second store required.

Metadata mapping: POMA fields → Weaviate primitives ​

POMA chunk fieldWeaviate primitiveWhat it enables
to_embedcontent TEXT property + self-provided vectorBM25F and vector halves of hybrid, same unit
file_idTEXT property, filterablescope queries to one document
page (on the archive's chunk records, from Docling item provenance; take the chunkset's leaf chunk)INT property, filterablepage-cited answers, page-range filters
depth of the leaf chunk (from explicit section_header levels)INT property, filterablefilter/re-rank by hierarchy level
chunkset_indexINT propertystable ordering at assembly time
chunk ↔ chunkset lineagecross-reference between collectionstraverse a hit to its full context in-database

Frequently asked questions ​

How do I get Docling output into Weaviate for RAG? ​

Save export_to_dict() as JSON, run PrimeCut().ingest() on it (auto-detected, tree parsed natively), insert chunksets into a collection with file_id/chunkset_index/page/depth properties and self-provided vectors — depth off the SDK objects, page from the .poma archive's chunk records for the chunkset's leaf chunk, then retrieve with collection.query.hybrid and assemble the cheatsheet.

What alpha should I use for Weaviate hybrid search over Docling-parsed documents? ​

Start at 0.5 and tune against your queries: lower weights BM25F for exact-term-heavy corpora, higher weights vectors for conversational questions. Chunksets serve both halves, since to_embed carries heading breadcrumbs alongside body text.

Do Docling's page headers and footers pollute Weaviate's BM25F scoring? ​

Only in the flatten-first pipeline, where repeated running titles become high-frequency term spam. POMA honors Docling's furniture classification and drops the noise before insertion, with its own header/footer strip on top.

How should I model chunks and chunksets in Weaviate collections? ​

One chunkset collection with filterable file_id/page/depth is enough for most stacks. For chunk-level granularity, add a Chunk collection cross-referencing its parent Chunkset — lineage traversal stays in-database.

Which Weaviate properties should be filterable for Docling content? ​

file_id (TEXT) for document scoping, page (INT) for page-cited answers, depth (INT) for hierarchy-aware filtering — with depth taken from Docling's explicit section_header levels, not re-inferred glyphs.

Same parser, different store: Docling → Qdrant · Docling → Pinecone · Docling → Chroma

Same store, different parser: Mistral OCR → Weaviate · LlamaParse → Weaviate · Azure Document Intelligence → Weaviate

Foundations: The Optimal Chunker for Docling · The Optimal Chunks for Weaviate · All pipeline recipes · RAG chunking guide