Skip to content

Docling Chunking for RAG: Chunkers, Tokenizers, Metadata ​

Docling chunking has two layers, and search results usually conflate them. The first layer is Docling's own chunker API: HierarchicalChunker, HybridChunker, PageChunker and LineBasedTokenChunker operate directly on the DoclingDocument and are genuinely structure-aware — they are the right answer if you want chunks sized to your embedding model's tokenizer. The second layer is what a retrieved passage carries when it comes back alone. Docling's chunks keep headings as strings in chunk.meta; POMA's PrimeCut takes the same DoclingDocument and returns chunksets — root-to-leaf paths through the heading tree, tables whole, page attribution kept.

This page covers both: the real Docling chunker API with working code, and what changes when the retrieval unit is a path instead of a window.

From DoclingDocument to chunksets: a Docling chunking example ​

Run your Docling conversion exactly as you do today, save the tree as JSON, and hand the file to PrimeCut. The shape is auto-detected.

bash
pip install docling poma
python
import json
from docling.document_converter import DocumentConverter
from poma import PrimeCut

# 1. Your existing Docling conversion — unchanged.
doc = DocumentConverter().convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

# 2. Chunk the DoclingDocument tree.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.docling.json")

print(len(result.chunks), len(result.chunksets))
print(result.chunksets[0].to_embed)  # embedding-ready text with its heading path

doc.save_as_json("contract.docling.json") works just as well; the file only has to contain the DoclingDocument. Because Docling already did the parsing, PrimeCut skips its own OCR front-end and runs the downstream stages only.

Docling's chunking methods, and the parameters that matter ​

Docling defines a chunker hierarchy on top of BaseChunker, whose contract is one abstract method, chunk(dl_doc: DoclingDocument) -> Iterator[BaseChunk], plus a concrete contextualize(chunk: BaseChunk) -> str that subclasses may override. Four implementations ship today.

ChunkerWhat it doesKey parameters
HierarchicalChunkerOne chunk per detected document element; merges list items by defaultmerge_list_items (deprecated upstream)
HybridChunkerHierarchical result, then tokenization-aware split of oversized chunks and merge of undersized peerstokenizer, merge_peers (default True), repeat_table_header (default True), omit_header_on_overflow (default False)
PageChunkerOne chunk per page; the whole document as one chunk when it carries no page informationserializer_provider
LineBasedTokenChunkerKeeps line boundaries intact — for tables, code, logs, liststokenizer, prefix, omit_prefix_on_overflow

Only the first two are re-exported from docling.chunking. The other two live in docling_core: from docling_core.transforms.chunker import PageChunker and from docling_core.transforms.chunker.line_chunker import LineBasedTokenChunker.

A complete Docling chunking example, using the tokenizer of the embedding model you will actually query with:

python
from docling.document_converter import DocumentConverter
from docling.chunking import HybridChunker
from docling_core.transforms.chunker.tokenizer.huggingface import HuggingFaceTokenizer
from transformers import AutoTokenizer

EMBED_MODEL_ID = "sentence-transformers/all-MiniLM-L6-v2"

doc = DocumentConverter().convert("contract.pdf").document

tokenizer = HuggingFaceTokenizer(
    tokenizer=AutoTokenizer.from_pretrained(EMBED_MODEL_ID),
    max_tokens=512,  # optional for HF: derived from the tokenizer when omitted
)
chunker = HybridChunker(tokenizer=tokenizer, merge_peers=True)

for chunk in chunker.chunk(dl_doc=doc):
    text_to_embed = chunker.contextualize(chunk=chunk)  # headings prepended
    print(chunk.meta.headings, len(text_to_embed))

Four things to know before you tune anything:

  • Embed contextualize(chunk=chunk), not chunk.text. chunk.text is the raw passage. contextualize() renders the metadata-enriched serialization — the form Docling intends you to feed an embedding model. Skipping it is the single most common Docling chunking bug.
  • Match the tokenizer to the embedding model. With docling-core[chunking] installed you get HuggingFaceTokenizer; with docling-core[chunking-openai], OpenAITokenizer(tokenizer=tiktoken.encoding_for_model("gpt-4o"), max_tokens=128*1024), where max_tokens is required. A mismatched tokenizer means the budget the chunker enforces is not the budget the model sees.
  • There is no overlap knob. Docling's chunkers expose no chunk_overlap parameter. Cuts land on structure, so neighbours do not need duplicated boundary text; merge_peers runs the opposite way, joining undersized successive chunks that share the same headings and captions.
  • Table headers repeat by default. When a table spans chunks, repeat_table_header=True re-emits the header row. omit_header_on_overflow=True drops it for a row that fits without the header but overflows with it.

Docling's docs: chunking concepts and the hybrid chunking recipe.

Why the DoclingDocument beats the markdown export ​

The other common Docling chunking path is export_to_markdown() into a generic text splitter. That discards the part Docling worked hardest for:

What the DoclingDocument holdsWhat markdown flattening leaves you
section_header items with a numeric level# glyphs, then cuts wherever the character count lands
tables as cell grids (data.table_cells)Pipe rows, sliced mid-table
pictures as positioned items with image.uriRefs, usually regex-stripped
page_header / page_footer items, and everything on the furniture content layerRunning titles and page numbers back in the text stream, then indexed as noise
prov[0].page_no per itemNo page attribution on any chunk

Keep the tree and the whole downstream chain gets better inputs. That holds whichever chunker you then run — Docling's own or PrimeCut's.

What actually travels with a chunk: Docling chunking metadata ​

A Docling chunk is a DocChunk whose meta is a DocMeta carrying doc_items (the source items, and through them the prov page numbers), headings as a list of strings, origin describing the source document, and a deprecated captions field. That is real, useful metadata, and contextualize() folds the headings into the embedded string.

It is also where the ceiling sits. Each chunk is still an isolated unit: the passage under Early termination arrives as one window, with ["Master Services Agreement", "Termination clauses"] attached as flat strings. Nothing downstream models the fact that these three levels are one path, that the sibling paragraphs belong to the same section, or that the table two items later completes the clause. At retrieval time the model reads one window and answers from it. For the full taxonomy of these failure modes, see the RAG chunking strategies guide.

PrimeCut's retrieval unit is the path itself. Chunks carry chunk_index, content, depth and file_id; chunksets group them into root-to-leaf paths, so chunkset.to_embed is a passage that already contains the chapter and section it lives under. No window size, no overlap.

One illustration of what that is worth: on a legal document in our reference set, the same question was answered from 337 tokens of retrieved context with chunksets versus 1,542 with a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.

How PrimeCut reads the DoclingDocument ​

The bring-your-own-OCR connector parses the tree, not the flattened markdown. What it does, item by item:

  1. Shape check up front. The payload is fingerprinted: a raw schema_name: "DoclingDocument", the docling-serve document.json_content wrapper, or an md_content envelope. If it confidently detects as a different engine's result than you declared, the request fails with a clear 422 instead of degrading the index quietly.
  2. Reading order from body.children. The walk is recursive and resolves groups, so list items nested inside a group container are never dropped, and a visited-set stops an item reachable twice from being emitted twice.
  3. Heading levels taken at face value. title becomes the top heading; a section_header becomes real #/##/### depth from its explicit level. Nothing Docling already decided gets re-inferred.
  4. Furniture preserved as metadata, not deleted. Items labeled page_header, page_footer or page_number, plus everything in the furniture layer, are kept out of the body and recorded as page artifacts in content_metadata — visible, not silently gone. POMA's own running-header/footer strip runs on top.
  5. Tables stay tables. A table item's data.table_cells grid is rebuilt as HTML, so an embedding never sees half a table. Caption refs are resolved and emitted next to their table or picture, once.
  6. Pictures handled, losses counted. A picture with an embedded data: URI goes through the shared image describe stage with its caption bound as alt text. A picture whose bytes live outside the payload is counted, so the loss shows up in content_metadata rather than vanishing.
  7. Pages, then hierarchy. Each item's prov[0].page_no partitions the parse by page before the heading tree is rebuilt into chunks and chunksets.

If your docling-serve response carries only md_content, POMA falls back to a markdown passthrough and runs the same downstream stages. Request the full tree when you can: explicit section_header levels beat re-inferred ones. To force or suppress detection, set external_ocr_source to "docling" or "none"; the API also exposes chunk_external_ocr_result for explicit routing (API reference).

Docling chunking options, compared ​

ApproachInputStructure usedRetrieval unitTablesFurniture
Recursive character splitterExported markdownNoneFixed-size fragmentCut mid-rowIndexed as noise
Markdown-header splitterExported markdown# glyphs onlyIsolated section fragmentUsually surviveIndexed as noise
HierarchicalChunkerDoclingDocumentOne chunk per elementSingle element, headings in metaWholeHandled by Docling
HybridChunkerDoclingDocumentElements + token budgetToken-sized window, headings in metaWhole, header repeatedHandled by Docling
PrimeCut chunksetsDoclingDocumentExplicit levels, full treeRoot-to-leaf pathHTML, never cutKept as page artifacts

Docling RAG: LangChain, and the pipeline end to end ​

For a Docling RAG pipeline in LangChain there are two supported routes. Docling's own is langchain-docling:

python
from langchain_docling.loader import DoclingLoader, ExportType
from docling.chunking import HybridChunker

loader = DoclingLoader(
    file_path="contract.pdf",
    export_type=ExportType.DOC_CHUNKS,  # or ExportType.MARKDOWN
    chunker=HybridChunker(tokenizer=tokenizer),
)
docs = loader.load()  # one LangChain Document per chunk

POMA's route takes the saved DoclingDocument JSON and yields one Document per chunkset, with its member chunks in metadata:

python
from poma import PrimeCut
from poma.integrations.langchain import PomaFileLoader, PomaChunksetSplitter

docs = PomaFileLoader("contract.docling.json").load()
chunkset_docs = PomaChunksetSplitter(client=PrimeCut()).split_documents(docs)

Without LangChain, the full path from PDF to answer — Qdrant here, but any store works:

bash
pip install docling 'poma[qdrant]'
python
import json, os
from docling.document_converter import DocumentConverter
from poma import PrimeCut
from poma.integrations.qdrant import PomaQdrant

# 1. Docling conversion, unchanged.
doc = DocumentConverter().convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

# 2. Chunk the tree.
result = PrimeCut().ingest("contract.docling.json")

# 3. Store: hybrid dense + sparse points, one call.
qdrant = PomaQdrant(
    url=os.environ["QDRANT_URL"],
    api_key=os.environ["QDRANT_API_KEY"],
    cloud_inference=False,  # False embeds locally via fastembed; True needs Qdrant Cloud inference
    collection_name="contracts",
    dense_model="sentence-transformers/all-MiniLM-L6-v2",
    sparse_model="Qdrant/bm25",
    dense_size=384,
    auto_create_collection=True,
)
qdrant.upsert_poma_points(result)

# 4. Retrieve prompt-ready context.
cheatsheets = qdrant.get_cheatsheets(query="What are the early termination conditions?", limit=3)
print(cheatsheets[0]["content"])

Each point's payload carries file_id, chunkset_index, the member chunks indices, the embedded text, and chunk_details with each chunk's content and depth — enough to reassemble a cheatsheet at query time without a second store.

Where Docling sits among document parsers ​

Worth saying plainly, because "docling vs X" searches usually want a decision, not a fight. Docling is one of the strongest open-source converters available: self-hostable, typed output, explicit heading levels, real table structure, page furniture separated for you, and a chunker API most parsers do not ship at all. If it fits your accuracy, cost and data-locality constraints, there is no reason to move.

The alternatives differ on deployment more than on ambition. Hosted engines — Mistral OCR, LlamaParse, Azure Document Intelligence, AWS Textract — trade self-hosting for managed scale. Marker and PaddleOCR-VL are the other common local options. PrimeCut ingests all of them, and parses PDFs, Office files, HTML and images itself when you would rather not run a parser at all. Switching parsers is rarely what fixes retrieval; what each retrieved unit carries usually is.

Frequently asked questions ​

What is the best Docling chunking strategy for RAG? ​

Chunk the DoclingDocument, never a markdown flattening of it. Inside Docling, HybridChunker is the strongest built-in option: it refines HierarchicalChunker output against your embedding tokenizer, splitting oversized chunks and merging undersized peers that share headings. Its retrieval unit is still one isolated passage with heading strings in metadata. PrimeCut takes the same tree and returns chunksets, root-to-leaf paths in which every leaf travels with its full heading path, so the retrieved unit is self-explanatory without any window tuning.

Docling chunking tokenizer and max_tokens: how do I set them? ​

Pass a tokenizer object to HybridChunker. For HuggingFace models, HuggingFaceTokenizer(tokenizer=AutoTokenizer.from_pretrained(EMBED_MODEL_ID), max_tokens=MAX_TOKENS); max_tokens is optional there and is derived from the tokenizer when omitted. For OpenAI embeddings, install docling-core[chunking-openai] and use OpenAITokenizer(tokenizer=tiktoken.encoding_for_model("gpt-4o"), max_tokens=128*1024), where max_tokens is required. Use the same tokenizer as your embedding model, otherwise the token budget the chunker enforces is not the budget the model sees.

Docling chunking overlap: is there a knob for it? ​

No. Docling's chunkers expose no chunk_overlap parameter. Overlap is a text-splitter workaround for cuts that land in the middle of a thought; Docling cuts on document structure instead, so consecutive chunks do not duplicate each other's boundary text. HybridChunker's merge_peers step, on by default, goes the other way and joins undersized neighbouring chunks that share the same headings. POMA chunksets need no overlap either: context is carried by the heading path, not by repeated sentences.

Docling chunking markdown export or DoclingDocument: which should I chunk? ​

You can, and it is the most common mistake in Docling RAG pipelines. export_to_markdown turns typed items into display text: explicit section_header levels become bare glyphs, table cell grids become pipes, picture items become refs, and the furniture group's page headers and footers are poured back into the text stream. Whatever splitter runs next has to re-guess structure Docling had already made explicit. Export the tree with export_to_dict and chunk that. POMA also accepts a markdown-only docling-serve payload through a passthrough, but the tree is the better input.

What metadata does a Docling chunk carry? ​

A Docling chunk is a DocChunk with a DocMeta: doc_items (the source items, which carry the prov page numbers), headings as a list of strings, an origin describing the source document, and a deprecated captions field. contextualize() renders that metadata into the string you embed. PrimeCut's units carry chunk_index, content, depth and file_id per chunk, with chunksets grouping chunks into root-to-leaf paths, and Docling's per-item prov page_no drives the page partitioning of the parse.

How do I build a Docling RAG pipeline with LangChain? ​

Two supported routes. Docling's own: pip install langchain-docling, then DoclingLoader(file_path=FILE_PATH, export_type=ExportType.DOC_CHUNKS, chunker=HybridChunker(tokenizer=tokenizer)), which yields one LangChain Document per chunk. POMA's: PomaFileLoader on the saved DoclingDocument JSON, then PomaChunksetSplitter(client=PrimeCut()).split_documents(docs), which yields one LangChain Document per chunkset with its member chunks in metadata. Both feed any LangChain vector store.

Can POMA ingest a DoclingDocument JSON directly? ​

Yes. Save export_to_dict() as JSON and pass the file to PrimeCut().ingest(). Detection fingerprints schema_name == "DoclingDocument", the docling-serve document.json_content wrapper, and the md_content envelope. You can also declare external_ocr_source: "docling" on the request, or set it to "none" to opt a look-alike JSON out. A payload that confidently detects as a different engine is rejected with a 422 rather than silently degrading the index.

How does Docling compare to other document parsers? ​

Docling is one of the strongest open-source, self-hostable converters: typed output, explicit heading levels, table structure, pre-separated page furniture, and built-in chunkers. Hosted engines such as Mistral OCR, LlamaParse and Azure Document Intelligence trade self-hosting for managed scale and their own strengths; Marker and PaddleOCR-VL are the other common local options. If Docling already fits your accuracy, cost and data-locality constraints, keep it. Retrieval quality is mostly decided by chunking and by what context each retrieved unit carries, not by swapping one strong parser for another.

From Docling to your vector database ​

End-to-end recipes — Docling result in, optimal retrieval out:

The rest of the series ​

The optimal chunker, for every OCR and parsing tool you already run:

Or start with the fundamentals: RAG chunking guide · Document ingestion guide · POMA chunksets.