Skip to content

PaddleOCR-VL to Elasticsearch: Chunking for RAG ​

The short answer: PaddleOCR-VL gives you self-hosted, layout-aware parsing; Elasticsearch gives you mature k-NN plus BM25 hybrid search. Wired together naively — dump markdown.text into a document, embed, index — they still produce mediocre RAG, because nothing in between rebuilds the cross-page hierarchy PP-DocLayoutV3 already detected. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (either shape, auto-detected), emits hierarchy-preserving chunksets, and vektoria writes them as content-free vectors into Elasticsearch's dense_vector mapping while the actual content lives on a volume. Retrieval runs a normal knn search and hands the result to assemble().

What each end of the pipeline actually provides ​

PaddleOCR-VL pairs a layout-detection model (PP-DocLayoutV3) with a vision-language OCR model, served behind your own /layout-parsing HTTP API — typically vLLM, self-hosted for data locality, air-gapped deployments, or per-page cost at volume. It returns either a PaddleX serving envelope (markdown text plus a prunedResult block list) or a raw save_to_json() block list (parsing_res_list, each block labeled doc_title, paragraph_title, table, or furniture). What it doesn't provide: a decision about what a retrieval unit should be. Details: The Optimal Chunker for PaddleOCR-VL.

Elasticsearch provides a dense_vector field for approximate k-NN, native BM25 on analyzed text fields, and a single request combining both via knn plus bool/match, fused server-side with rank. What it doesn't provide: any opinion about what belongs in that document. It ranks whatever vector and text you indexed — including HTML-polluted fragments, if that's what you gave it. Details: Elasticsearch Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — read markdown.text (or flatten parsing_res_list block by block), embed the whole page, index it as one Elasticsearch document with text holding that same markdown — breaks in a pair-specific way: PaddleOCR-VL's table blocks are already inline HTML, by design, so the analyzed content going into Elasticsearch's text field carries raw markup. Elasticsearch's default standard analyzer does not strip HTML tags before tokenizing, so table, tr, td, and attribute fragments get indexed as ordinary terms. Every BM25 query now competes against tag-name noise repeated across every page that has a table — diluting precision on exactly the hybrid signal you added Elasticsearch for. A chunker that separates display content (which can keep HTML) from embed/index text (which shouldn't) avoids this entirely.

The pipeline, end to end ​

bash
pip install elasticsearch
python
import json
import requests
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from elasticsearch import Elasticsearch

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("result.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. Chunk the raw result and keep the .poma archive — the volume's source of truth.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("result.paddleocr-vl.json", download_dir="archives", filename="doc.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
es = Elasticsearch("http://localhost:9200")  # local dev cluster (xpack.security.enabled=false); a secured cluster needs https + basic_auth/api_key + ca_certs

es.indices.create(index="poma", mappings={"properties": {
    "embedding": {"type": "dense_vector", "dims": embedder.dims, "index": True, "similarity": "cosine"},
    "file_id": {"type": "keyword"},
    "chunkset_index": {"type": "integer"},
    "text": {"type": "text"},
}}, ignore=400)

# 3. Content-free ingest — vector index holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/doc.poma")  # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/doc.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    es.index(index="poma", id=r.id, document={
        "embedding": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
        "text": r.text,
    })

# 4. Hybrid k-NN retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
es.indices.refresh(index="poma")  # make the just-indexed docs searchable before querying
res = es.search(index="poma", knn={"field": "embedding", "query_vector": qv,
                                    "k": 10, "num_candidates": 100})
context = assemble(res.body, volume=vol)  # elasticsearch-py 8+ wraps the response (ObjectApiResponse) — pass the dict body  # -> [{"file_id", "content"}, ...]

PrimeCut validates the payload up front (a corrupted or mislabeled upload 422s immediately), detects PaddleOCR-VL from layoutParsingResults or parsing_res_list — both are the tool's own key names — strips running furniture, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none".

Metadata mapping: POMA fields → Elasticsearch primitives ​

POMA fieldSource in the PaddleOCR-VL resultElasticsearch primitiveWhat it enables
to_embed (chunkset text)markdown.text when present, else assembled from parsing_res_list in block_orderembedding, a dense_vector fieldk-NN vector search signal
file_idone per ingested layout-parsing JSONkeyword field on the indexscope knn/bool queries to one document
chunkset_indexassigned by PrimeCut while rebuilding hierarchyinteger field, half of the deterministic point iddedupe + volume lookup inside assemble()
page (from page_index)0-based in both PaddleOCR-VL shapes, remapped 1-basedkept in the volume's chunk payload, not indexedpage-cited cheatsheets without bloating the index
heading depth / lineagerebuilt from doc_title/paragraph_title across pageskept in the volume's chunk payloadhierarchy-aware citations at assemble() time

Retrieved chunksets share ancestor lineage across a document — assemble() deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answers our reference legal-document benchmark with 337 tokens instead of 1,542 for a recursive-splitter baseline. Methodology here.

Frequently asked questions ​

How do I get self-hosted PaddleOCR-VL results into Elasticsearch for RAG? ​

Save the raw layout-parsing JSON, run PrimeCut().ingest() on it, then use records_from_archive to embed and upsert content-free vectors with file_id/chunkset_index into a dense_vector mapping. Retrieve with a knn search and assemble().

Why does concatenating PaddleOCR-VL markdown into one Elasticsearch document hurt BM25 relevance? ​

Table blocks are already inline HTML, and Elasticsearch's default analyzer doesn't strip markup — tag names get indexed as terms, diluting BM25 precision. Chunking separates display content from embed text so this never reaches the analyzed field.

What Elasticsearch mapping fields should PaddleOCR-VL chunksets carry? ​

embedding as dense_vector, file_id and chunkset_index as keyword/integer. Page and depth stay in the volume's chunk payload, not the index.

Do PaddleOCR-VL's inline HTML tables cause problems in an Elasticsearch text field? ​

Only if you index raw markdown directly. PrimeCut keeps table HTML as display content on the volume and embeds a normalized to_embed text instead.

Does self-hosting PaddleOCR-VL change how you should query Elasticsearch? ​

No — the knn clause and hybrid rank fusion work identically. What changes is upstream: documents never leave your infrastructure through the OCR step.

Same parser, different store: PaddleOCR-VL → OpenSearch · PaddleOCR-VL → Redis · PaddleOCR-VL → Qdrant

Same store, different parser: Mistral OCR → Elasticsearch · Docling → Elasticsearch

Foundations: The Optimal Chunker for PaddleOCR-VL · Elasticsearch Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide