Skip to content

Docling to OpenSearch: Chunking for RAG ​

The short answer: Docling gives you a typed DoclingDocument tree in which tables are cell grids, not markdown approximations; OpenSearch gives you a knn_vector mapping and a hybrid search pipeline that fuses lexical and vector scores. Wired together naively — flatten to markdown, split by character count, index — a table Docling kept whole gets cut mid-row into two documents, and OpenSearch's hybrid normalization can surface both fragments instead of the complete table. The missing link is POMA: PrimeCut parses the DoclingDocument tree into chunksets that keep tables intact, and poma.vektoria indexes them as content-free documents — (id, vector, {file_id, chunkset_index}) — with the real text on a volume. Retrieval runs OpenSearch's own knn query, then assemble() returns prompt-ready cheatsheets.

What each end of the pipeline actually provides ​

Docling (IBM's open-source converter, also served via docling-serve) returns the DoclingDocument: a typed texts/tables/pictures/groups tree in which table items carry structured cell grids, section_header items carry an explicit numeric level, and repeating page furniture is parked in the furniture group. What it doesn't provide: retrieval units, or a schema for whatever search engine receives them. Details: The Optimal Chunker for Docling.

OpenSearch provides knn_vector fields backed by HNSW via nmslib or faiss, native BM25 on analyzed text fields, and a dedicated hybrid query type that normalizes and fuses lexical and vector sub-query scores through a search pipeline — distinct from Elasticsearch's RRF rank since the 2021 fork. What it doesn't provide: any opinion about what a document should contain. Details: OpenSearch Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — doc.export_to_markdown() → character splitter → osc.index(...) — breaks in a pair-specific way that's easy to miss until a table-heavy query comes back wrong.

Docling keeps a table as one typed table item with a full cell grid, already reassembled from whatever columns and rows the source layout used. export_to_markdown() serializes that grid to pipe syntax as a single block, but a character splitter downstream knows nothing about row boundaries — it cuts wherever the token or character count lands, frequently mid-row. Both halves are embedded and indexed as separate knn_vector documents, near-identical to each other because they share most of the same table's tokens. OpenSearch's hybrid query type then normalizes lexical and vector scores across the result set before fusing them, and two fragments that score and normalize similarly can both surface in the fused top-k — so a query about a number in that table returns the same half-table twice, with the other half missing, instead of the one complete table PrimeCut would have kept as a single chunk.

The pipeline, end to end ​

bash
pip install docling opensearch-py poma
python
import json
from docling.document_converter import DocumentConverter
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from opensearchpy import OpenSearch

# 1. Docling conversion — your existing call, unchanged.
converter = DocumentConverter()
doc = converter.convert("contract.pdf").document
with open("contract.docling.json", "w") as f:
    json.dump(doc.export_to_dict(), f)

# 2. The missing link — DoclingDocument tree in, .poma archive out; tables stay whole.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("contract.docling.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
osc = OpenSearch(hosts=[{"host": "localhost", "port": 9200}])

osc.indices.create(index="poma", body={"settings": {"index.knn": True},  # knn_vector fields need index.knn=true, otherwise the mapping is rejected (and ignore=400 hides it)
                                       "mappings": {"properties": {
    "embedding": {"type": "knn_vector", "dimension": embedder.dims,
                  "method": {"name": "hnsw", "space_type": "cosinesimil", "engine": "faiss"}},
    "file_id": {"type": "keyword"},
    "chunkset_index": {"type": "integer"},
    "text": {"type": "text"},
}}}, ignore=400)

# 3. Content-free ingest — index holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    osc.index(index="poma", id=r.id, body={
        "embedding": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
        "text": r.text,
    })

# 4. Retrieval — knn returns the document body by default, then assemble.
qv = embedder.embed(["What are the early termination conditions?"])[0]
osc.indices.refresh(index="poma")  # make the just-indexed docs searchable before querying
res = osc.search(index="poma", body={"query": {"knn": {"embedding": {
    "vector": qv, "k": 10}}}})
context = assemble(res, volume=vol)  # -> [{"file_id", "content"}, ...]
print(context[0]["content"])

Auto-detection fingerprints schema_name == "DoclingDocument" up front — a corrupted or mislabeled upload 422s immediately, never silently degrading downstream. A docling-serve envelope carrying only md_content falls back to a markdown passthrough. On our reference legal-document benchmark, this pipeline answers with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with zero information loss — methodology in Document Ingestion & Chunking for RAG.

Metadata mapping: POMA fields → OpenSearch primitives ​

POMA fieldOpenSearch primitiveWhat it enables
r.id (deterministic chunkset_uuid)document _idsame chunkset lands on the same document across every connector
r.text (the to_embed text, tables kept whole)knn_vector field + analyzed text fieldknn + BM25 fused via the hybrid query type
file_idkeyword fieldexact-match filtering scoped to one document
chunkset_indexinteger fieldreturned in the document body by default — no extra flag for assemble()
chunks (member chunk list, including Docling table and section_header items)written to the volume document, not an index fieldcheatsheet and full-table reconstruction without widening the mapping

Frequently asked questions ​

How do I get Docling output into OpenSearch for RAG? ​

Save export_to_dict() as JSON, run it through PrimeCut (which keeps tables intact), map knn_vector + keyword file_id + integer chunkset_index, then index via records_from_archive(). Retrieve with knn, then assemble().

What happens to a Docling table if I split its markdown export instead of using PrimeCut? ​

The cell grid gets serialized to markdown and cut mid-row by a character splitter, producing two near-identical fragment documents instead of one complete table chunk.

How does OpenSearch's hybrid search pipeline behave when two hits are fragments of the same split table? ​

Its search pipeline normalizes and fuses lexical and vector scores; two similarly-scoring table fragments can both surface in the fused top-k, returning the same half-table twice.

Is OpenSearch's knn_vector mapping the same as Elasticsearch's dense_vector for a Docling pipeline? ​

Conceptually close, not identical — OpenSearch's own k-NN plugin (knn_vector, nmslib/faiss) diverged from Elasticsearch's dense_vector after the 2021 fork. The retrieval one-liner behaves the same; the mapping syntax doesn't.

Does OpenSearch return enough on a knn hit for POMA's assemble() to work? ​

Yes — the document body returns by default, so file_id and chunkset_index come back with no extra request parameter.

Same parser, different store: Docling → Elasticsearch · Docling → Vespa · Docling → pgvector

Foundations: The Optimal Chunker for Docling · OpenSearch Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide