PaddleOCR-VL to OpenSearch: Chunking for RAG
The short answer: PaddleOCR-VL gives you self-hosted, layout-aware parsing; OpenSearch gives you a mature, independently-forked k-NN plugin plus BM25. Wired together naively — read whichever shape happens to come back, embed the page, index it — they still produce mediocre RAG, because nothing in between rebuilds the cross-page hierarchy PP-DocLayoutV3 already detected, and nothing checks that the shape you handled is the shape you got. The missing link is POMA: PrimeCut().ingest() consumes the raw layout-parsing JSON (either shape, auto-detected), emits hierarchy-preserving chunksets, and vektoria writes them as content-free vectors into OpenSearch's knn_vector mapping while the actual content lives on a volume. Retrieval runs a normal k-NN search and hands the result to assemble().
What each end of the pipeline actually provides
PaddleOCR-VL pairs a layout-detection model (PP-DocLayoutV3) with a vision-language OCR model, served behind your own /layout-parsing HTTP API — typically vLLM, self-hosted for data locality, air-gapped deployments, or per-page cost at volume. It returns either a PaddleX serving envelope (a markdown object with reading-ordered text) or a raw save_to_json() block list (parsing_res_list, no markdown key at all). A pipeline that assumes only one shape will always exist is guessing. Details: The Optimal Chunker for PaddleOCR-VL.
OpenSearch provides a knn_vector field (HNSW via faiss or nmslib), native BM25, and a dedicated hybrid query type that normalizes and fuses both signals through a search pipeline — its own mechanism, distinct from Elasticsearch's RRF. What it doesn't provide: validation that the document you indexed actually contains anything. An empty text field with a valid vector indexes and queries exactly like a real one. Details: OpenSearch Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — branch ingestion code on the PaddleX serving envelope's markdown.text, embed it, index it — breaks the moment a batch instead comes from the offline .save_to_json() pipeline method, which never emits a markdown key at all, only parsing_res_list. A script with no fallback branch either throws deep in a .get() chain or, worse, silently writes an OpenSearch document with an empty text field and a vector embedded from an empty string. OpenSearch has no schema rule requiring text to be non-empty, so indexing succeeds, the k-NN query returns the point as a legitimate hit, and the only visible symptom is a blank cheatsheet at generation time — long after the ingestion job reported success.
The pipeline, end to end
pip install opensearch-pyimport json
import requests
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from opensearchpy import OpenSearch
# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
# PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
"http://layout-server:40111/layout-parsing",
json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("result.paddleocr-vl.json", "w") as f:
json.dump(resp.json(), f)
# 2. Chunk the raw result and keep the .poma archive — the volume's source of truth.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("result.paddleocr-vl.json", download_dir="archives", filename="doc.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
osc = OpenSearch(hosts=[{"host": "localhost", "port": 9200}])
osc.indices.create(index="poma", body={"settings": {"index.knn": True}, # knn_vector fields need index.knn=true, otherwise the mapping is rejected (and ignore=400 hides it)
"mappings": {"properties": {
"embedding": {"type": "knn_vector", "dimension": embedder.dims,
"method": {"name": "hnsw", "space_type": "cosinesimil", "engine": "faiss"}},
"file_id": {"type": "keyword"},
"chunkset_index": {"type": "integer"},
"text": {"type": "text"},
}}}, ignore=400)
# 3. Content-free ingest — vector index holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/doc.poma") # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/doc.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
osc.index(index="poma", id=r.id, body={
"embedding": embedder.embed([r.text])[0],
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
"text": r.text,
})
# 4. k-NN retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
osc.indices.refresh(index="poma") # make the just-indexed docs searchable before querying
res = osc.search(index="poma", body={"query": {"knn": {"embedding": { # opensearch-py has no `knn=` kwarg (that is the Elasticsearch client); k-NN is a query clause
"vector": qv, "k": 10}}}})
context = assemble(res, volume=vol) # -> [{"file_id", "content"}, ...]PrimeCut never has this problem: its detector fingerprints layoutParsingResults or parsing_res_list — both are PaddleOCR-VL's own key names — and handles both shapes in the same code path, so a save_to_json()-only batch chunks exactly as correctly as a serving-envelope batch. To force or suppress detection, set external_ocr_source to "paddleocr_vl" or "none".
Metadata mapping: POMA fields → OpenSearch primitives
| POMA field | Source in the PaddleOCR-VL result | OpenSearch primitive | What it enables |
|---|---|---|---|
to_embed (chunkset text) | markdown.text when present, else assembled from parsing_res_list in block_order | embedding, a knn_vector field | k-NN vector search signal |
file_id | one per ingested layout-parsing JSON | keyword field on the index | scope knn/hybrid queries to one document |
chunkset_index | assigned by PrimeCut while rebuilding hierarchy | integer field, half of the deterministic point id | dedupe + volume lookup inside assemble() |
page (from page_index) | 0-based in both PaddleOCR-VL shapes, remapped 1-based | kept in the volume's chunk payload, not indexed | page-cited cheatsheets without bloating the index |
| heading depth / lineage | rebuilt from doc_title/paragraph_title across pages | kept in the volume's chunk payload | hierarchy-aware citations at assemble() time |
Retrieved chunksets share ancestor lineage across a document — assemble() deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answers our reference legal-document benchmark with 337 tokens instead of 1,542 for a recursive-splitter baseline. Methodology here.
Frequently asked questions
How do I get self-hosted PaddleOCR-VL results into OpenSearch for RAG?
Save the raw layout-parsing JSON, run PrimeCut().ingest() on it, then use records_from_archive to embed and upsert content-free vectors with file_id/chunkset_index into a knn_vector mapping. Retrieve with a knn search and assemble().
What happens if my ingestion script only handles one of PaddleOCR-VL's two output shapes?
It silently produces blank documents when a batch comes from the shape you didn't handle — OpenSearch doesn't validate that text is non-empty, so it indexes fine and only an empty cheatsheet at generation time reveals the gap.
What OpenSearch mapping fields should PaddleOCR-VL chunksets carry?
embedding as knn_vector (HNSW via faiss/nmslib), file_id and chunkset_index as keyword/integer. Page and depth stay in the volume's chunk payload, not the index.
Is OpenSearch retrieval for PaddleOCR-VL chunks different from Elasticsearch?
Chunking is identical. The index differs: OpenSearch's knn_vector field and hybrid search pipeline are distinct from Elasticsearch's dense_vector and RRF rank.
Does self-hosting PaddleOCR-VL change how you should query OpenSearch?
No — the k-NN plugin works identically. What changes is upstream: documents never leave your infrastructure through the OCR step.
Related recipes
Same parser, different store: PaddleOCR-VL → Elasticsearch · PaddleOCR-VL → Redis · PaddleOCR-VL → Chroma
Same store, different parser: LlamaParse → OpenSearch · Docling → OpenSearch
Foundations: The Optimal Chunker for PaddleOCR-VL · OpenSearch Chunking Strategy for RAG · All pipeline recipes · RAG architecture guide