Skip to content

Docling to Vespa: Chunking for RAG ​

The short answer: Docling gives you a typed DoclingDocument tree with explicit heading levels and pre-isolated page furniture; Vespa gives you tensor fields, native BM25, and multi-phase ranking inside one document type. Wired together naively — export_to_markdown(), split, feed — Vespa's rank profile ends up scoring exactly the noise Docling had already isolated for free. The missing link is POMA: PrimeCut parses the DoclingDocument tree into chunksets, and poma.vektoria feeds them as content-free Vespa documents — (id, vector, {file_id, chunkset_index}) — with the real text on a volume. Retrieval runs Vespa's own YQL query, then assemble() returns prompt-ready cheatsheets.

What each end of the pipeline actually provides ​

Docling (IBM's open-source converter, also served via docling-serve) returns the DoclingDocument: a typed texts/tables/pictures/groups tree in which section_header items carry an explicit numeric level, tables are cell grids rather than pipe approximations, and repeating page furniture is parked in the furniture group with page_header/page_footer labels. What it doesn't provide: retrieval units, or an opinion about how those units should be scored. Details: The Optimal Chunker for Docling.

Vespa stores vectors as ordinary tensor fields inside the same document as scalar attributes and BM25-indexed text — no separate vector-store round trip. A single YQL query combines userQuery() with nearestNeighbor(), fused by a rank profile using multi-phase ranking. What it doesn't provide: any opinion about what belongs in that document. Details: Vespa Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — doc.export_to_markdown() → character splitter → feed → default rank profile — breaks in a pair-specific way, because Vespa's BM25 is scored across the whole content cluster, not per document.

Docling already tells you what's noise: the furniture group and its page_header/page_footer labels are pre-isolated, deterministic, and free — your precursor did the classification. Flatten to markdown first, and that classification disappears; a repeating footer like "Confidential — page 41" gets fed into the same index: enable-bm25 field as real content. Because bm25(text) is a corpus-wide statistic, not a per-document one, this boilerplate — appearing on nearly every page of every document — accumulates term frequency the rank profile's closeness(...) * (1 + bm25(...)) formula then rewards, surfacing furniture above the passage that actually answers the question. At the same time, the explicit section_header.level that would have populated a depth attribute is gone, so there's no hierarchy signal left to filter or re-rank by at all.

The pipeline, end to end ​

bash
pip install docling pyvespa poma
python
import json
from docling.document_converter import DocumentConverter
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from vespa.package import ApplicationPackage, Field, Schema, Document, HNSW, RankProfile
from vespa.deployment import VespaDocker

# 1. Docling conversion — your existing call, unchanged.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

# 2. The missing link — DoclingDocument tree in, .poma archive out.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.docling.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")

package = ApplicationPackage(
    name="contracts",
    schema=[Schema(
        name="chunkset",
        document=Document(fields=[
            Field(name="file_id", type="string", indexing=["attribute"]),
            Field(name="chunkset_index", type="int", indexing=["attribute"]),
            Field(name="text", type="string", indexing=["index"], index="enable-bm25"),
            Field(
                name="embedding", type=f"tensor<float>(x[{embedder.dims}])",
                indexing=["attribute", "index"], ann=HNSW(distance_metric="angular"),
            ),
        ]),
        rank_profiles=[RankProfile(
            name="fusion", inputs=[("query(q)", f"tensor<float>(x[{embedder.dims}])")],
            first_phase="closeness(field, embedding) * (1 + bm25(text))",
        )],
    )],
)
app = VespaDocker().deploy(application_package=package)

# 3. Content-free feed — document fields hold routing metadata only.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
docs = []
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    docs.append({"id": r.id, "fields": {
        "file_id": r.payload["file_id"], "chunkset_index": r.payload["chunkset_index"],
        "text": r.text, "embedding": embedder.embed([r.text])[0],
    }})
app.feed_iterable(docs, schema="chunkset", namespace="contracts")

# 4. Retrieval — select file_id/chunkset_index explicitly, then assemble.
qv = embedder.embed(["What are the early termination conditions?"])[0]
resp = app.query(
    yql="select file_id, chunkset_index from sources * where userQuery() or "
        "({targetHits:1000}nearestNeighbor(embedding, q))",
    query="early termination conditions", ranking="fusion",
    body={"input.query(q)": qv},
)
context = assemble(resp, volume=vol)  # -> [{"file_id", "content"}, ...]
print(context[0]["content"])

Auto-detection fingerprints schema_name == "DoclingDocument" up front — a corrupted or mislabeled upload 422s immediately, never silently degrading downstream. A docling-serve envelope carrying only md_content falls back to a markdown passthrough. On our reference legal-document benchmark, this pipeline answers with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss — methodology in Document Ingestion & Chunking for RAG.

Metadata mapping: POMA fields → Vespa primitives ​

POMA fieldVespa primitiveWhat it enables
r.id (deterministic chunkset_uuid)document idsame chunkset lands on the same document across every connector
r.text (the to_embed text)tensor field (HNSW) + index: enable-bm25 string fieldmulti-phase rank profile fusing ANN and BM25 in one query
file_idattribute fieldscope YQL predicates to one document
chunkset_indexattribute fieldmust be select-ed explicitly in YQL for assemble() to resolve content
chunks (member chunk list, including Docling section_header depth)written to the volume document, not a schema fieldcheatsheet reconstruction without widening the document type

Frequently asked questions ​

How do I get Docling output into Vespa for RAG? ​

Save export_to_dict() as JSON, run it through PrimeCut, then feed chunksets as Vespa documents with file_id/chunkset_index attributes and a tensor embedding via records_from_archive(). Retrieve with a YQL query selecting those fields explicitly, then assemble().

Why does my Vespa YQL query need to select fields explicitly for POMA to work? ​

assemble() needs file_id and chunkset_index on each hit. Vespa doesn't return arbitrary fields unless the YQL statement selects them — omit the explicit select and assemble() has nothing to resolve.

What Vespa schema fields should Docling chunksets populate? ​

file_id/chunkset_index as attributes, a BM25-enabled text field, and a tensor field with HNSW. Docling's section_header.level is the natural source for an optional depth attribute.

Does Vespa's rank profile care whether Docling's furniture was stripped before feeding? ​

Yes — bm25(text) is scored corpus-wide. Unstripped furniture repeating across every page accumulates term frequency the rank profile rewards, ranking boilerplate above real content.

Should I flatten the DoclingDocument to markdown before feeding Vespa? ​

No. Flattening collapses explicit heading levels and re-injects furniture into the text stream. Feed the tree; PrimeCut keeps levels as depth and honors Docling's furniture classification first.

Same parser, different store: Docling → Turbopuffer · Docling → OpenSearch · Docling → Pinecone

Foundations: The Optimal Chunker for Docling · Vespa Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide