Skip to content

PaddleOCR-VL to OpenSearch: Chunking for RAG ​

The short answer: PaddleOCR-VL gives you self-hosted, layout-aware parsing; OpenSearch gives you a mature, independently-forked k-NN plugin plus BM25. Wired together naively — read whichever shape happens to come back, embed the page, index it — they still produce mediocre RAG, because nothing in between rebuilds the cross-page hierarchy PP-DocLayoutV3 already detected, and nothing checks that the shape you handled is the shape you got. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (either shape, auto-detected), emits hierarchy-preserving chunksets, and vektoria writes them as content-free vectors into OpenSearch's knn_vector mapping while the actual content lives on a volume. Retrieval runs a normal k-NN search and hands the result to assemble().

What each end of the pipeline actually provides ​

PaddleOCR-VL pairs a layout-detection model (PP-DocLayoutV3) with a vision-language OCR model, served behind your own /layout-parsing HTTP API — typically vLLM, self-hosted for data locality, air-gapped deployments, or per-page cost at volume. It returns either a PaddleX serving envelope (a markdown object with reading-ordered text) or a raw save_to_json() block list (parsing_res_list, no markdown key at all). A pipeline that assumes only one shape will always exist is guessing. Details: The Optimal Chunker for PaddleOCR-VL.

OpenSearch provides a knn_vector field (HNSW via faiss or nmslib), native BM25, and a dedicated hybrid query type that normalizes and fuses both signals through a search pipeline — its own mechanism, distinct from Elasticsearch's RRF. What it doesn't provide: validation that the document you indexed actually contains anything. An empty text field with a valid vector indexes and queries exactly like a real one. Details: OpenSearch Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — branch ingestion code on the PaddleX serving envelope's markdown.text, embed it, index it — breaks the moment a batch instead comes from the offline .save_to_json() pipeline method, which never emits a markdown key at all, only parsing_res_list. A script with no fallback branch either throws deep in a .get() chain or, worse, silently writes an OpenSearch document with an empty text field and a vector embedded from an empty string. OpenSearch has no schema rule requiring text to be non-empty, so indexing succeeds, the k-NN query returns the point as a legitimate hit, and the only visible symptom is a blank cheatsheet at generation time — long after the ingestion job reported success.

The pipeline, end to end ​

bash
pip install opensearch-py
python
import json
import requests
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from opensearchpy import OpenSearch

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("result.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. Chunk the raw result and keep the .poma archive — the volume's source of truth.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("result.paddleocr-vl.json", download_dir="archives", filename="doc.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
osc = OpenSearch(hosts=[{"host": "localhost", "port": 9200}])

osc.indices.create(index="poma", body={"settings": {"index.knn": True},  # knn_vector fields need index.knn=true, otherwise the mapping is rejected (and ignore=400 hides it)
                                       "mappings": {"properties": {
    "embedding": {"type": "knn_vector", "dimension": embedder.dims,
                  "method": {"name": "hnsw", "space_type": "cosinesimil", "engine": "faiss"}},
    "file_id": {"type": "keyword"},
    "chunkset_index": {"type": "integer"},
    "text": {"type": "text"},
}}}, ignore=400)

# 3. Content-free ingest — vector index holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/doc.poma")  # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/doc.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    osc.index(index="poma", id=r.id, body={
        "embedding": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
        "text": r.text,
    })

# 4. k-NN retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
osc.indices.refresh(index="poma")  # make the just-indexed docs searchable before querying
res = osc.search(index="poma", body={"query": {"knn": {"embedding": {  # opensearch-py has no `knn=` kwarg (that is the Elasticsearch client); k-NN is a query clause
    "vector": qv, "k": 10}}}})
context = assemble(res, volume=vol)  # -> [{"file_id", "content"}, ...]

PrimeCut never has this problem: its detector fingerprints layoutParsingResults or parsing_res_list — both are PaddleOCR-VL's own key names — and handles both shapes in the same code path, so a save_to_json()-only batch chunks exactly as correctly as a serving-envelope batch. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none".

Metadata mapping: POMA fields → OpenSearch primitives ​

POMA fieldSource in the PaddleOCR-VL resultOpenSearch primitiveWhat it enables
to_embed (chunkset text)markdown.text when present, else assembled from parsing_res_list in block_orderembedding, a knn_vector fieldk-NN vector search signal
file_idone per ingested layout-parsing JSONkeyword field on the indexscope knn/hybrid queries to one document
chunkset_indexassigned by PrimeCut while rebuilding hierarchyinteger field, half of the deterministic point iddedupe + volume lookup inside assemble()
page (from page_index)0-based in both PaddleOCR-VL shapes, remapped 1-basedkept in the volume's chunk payload, not indexedpage-cited cheatsheets without bloating the index
heading depth / lineagerebuilt from doc_title/paragraph_title across pageskept in the volume's chunk payloadhierarchy-aware citations at assemble() time

Retrieved chunksets share ancestor lineage across a document — assemble() deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answers our reference legal-document benchmark with 337 tokens instead of 1,542 for a recursive-splitter baseline. Methodology here.

Frequently asked questions ​

How do I get self-hosted PaddleOCR-VL results into OpenSearch for RAG? ​

Save the raw layout-parsing JSON, run PrimeCut().ingest() on it, then use records_from_archive to embed and upsert content-free vectors with file_id/chunkset_index into a knn_vector mapping. Retrieve with a knn search and assemble().

What happens if my ingestion script only handles one of PaddleOCR-VL's two output shapes? ​

It silently produces blank documents when a batch comes from the shape you didn't handle — OpenSearch doesn't validate that text is non-empty, so it indexes fine and only an empty cheatsheet at generation time reveals the gap.

What OpenSearch mapping fields should PaddleOCR-VL chunksets carry? ​

embedding as knn_vector (HNSW via faiss/nmslib), file_id and chunkset_index as keyword/integer. Page and depth stay in the volume's chunk payload, not the index.

Is OpenSearch retrieval for PaddleOCR-VL chunks different from Elasticsearch? ​

Chunking is identical. The index differs: OpenSearch's knn_vector field and hybrid search pipeline are distinct from Elasticsearch's dense_vector and RRF rank.

Does self-hosting PaddleOCR-VL change how you should query OpenSearch? ​

No — the k-NN plugin works identically. What changes is upstream: documents never leave your infrastructure through the OCR step.

Same parser, different store: PaddleOCR-VL → Elasticsearch · PaddleOCR-VL → Redis · PaddleOCR-VL → Chroma

Same store, different parser: LlamaParse → OpenSearch · Docling → OpenSearch

Foundations: The Optimal Chunker for PaddleOCR-VL · OpenSearch Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide