Marker to OpenSearch: Chunking for RAG
The short answer: Marker gives you fast, local PDF parsing with a real Document block tree — per-page blocks, explicit types, and full HTML tables with spans intact, when you run its JSON renderer. OpenSearch gives you its own knn_vector field type and k-NN plugin (forked from Elasticsearch in 2021), plus a dedicated hybrid query type for fusing lexical and vector search. Wired together naively, Marker's page boundaries and table structure are gone before a document is indexed, and any lexical layer added on top ranks the resulting garbage tokens without the normalization OpenSearch's hybrid pipeline is built to provide. The missing link is POMA: PrimeCut().ingest() turns the saved Marker JSON into hierarchy-preserving chunksets, and poma.vektoria indexes them as content-free documents — embedding, file_id, chunkset_index — while the actual chunkset text lives on a volume. Retrieval runs OpenSearch's own knn query, then assemble() reconstructs prompt-ready cheatsheets.
What each end of the pipeline actually provides
Marker (by Datalab / VikParuchuri) runs on your own hardware and turns PDFs into clean structured output. With the JSON renderer (--output_format json) you get a Document block tree — Page children, per-block HTML, and full <table> elements with row and column spans intact. What it doesn't provide: cross-page hierarchy, retrieval units, or inline images — figure bytes live in a side-channel images dict keyed by name, with only  references left in the text. Details: The Optimal Chunker for Marker.
OpenSearch provides knn_vector fields with HNSW-backed approximate k-NN via nmslib or faiss, native BM25 on any analyzed text field, and a dedicated hybrid query type that fuses lexical and vector sub-queries through a search pipeline's score normalization — a distinct mechanism from Elasticsearch's RRF rank, since the two projects diverged at Elasticsearch 7.10 in 2021. What it doesn't provide: any opinion about what a document should represent — a mapping whose fields hold a broken table fragment or a dangling image reference gets ranked exactly as confidently as a clean one. Details: OpenSearch Chunking Strategy for RAG.
The naive wiring, and where it breaks
The common recipe — concatenate Marker's per-page markdown into one string, RecursiveCharacterTextSplitter, embed, osc.index() the raw chunk text — breaks in ways specific to this pair:
- Page boundaries disappear before a single document is indexed. Marker's markdown output alone has no page delimiters, so a flatten-then-split pipeline can't populate a
pagefield to filter or cite by — only the JSON renderer's Page blocks carry that boundary, and only if the ingest path reads them. - Tables get cut mid-row. The JSON renderer's tables arrive as full
<table>elements with colspans — real structure Marker recovered. A character splitter run over the flattened markdown routinely slices a table in half, leaving raw HTML tag fragments behind. - If a hybrid layer is added naively, unnormalized noise wins. OpenSearch's hybrid query type requires a search pipeline to normalize BM25 and vector scores before combining them. Leftover
<td>/<tr>tag text from a cut table, or danglingreferences, become unmatched or generic BM25 tokens; without normalization, an unbounded lexical score built on that noise can outrank a properly-scaled vector hit. - Every figure vanishes. Marker's images live in the side-channel
imagesdict; the markdown carries onlyreferences, which a markdown-only pipeline embeds as literal, unmatched text — the figure itself was never indexed.
The pipeline, end to end
pip install opensearch-py
# Your existing Marker run — unchanged. The JSON renderer keeps the
# Document block tree that POMA auto-detects, tables and pages included.
marker_single contract.pdf --output_format json --output_dir out/from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from opensearchpy import OpenSearch
# 1. Chunk the raw Marker JSON and keep the .poma archive — the volume's source of truth.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("out/contract/contract.json", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
osc = OpenSearch(hosts=[{"host": "localhost", "port": 9200}])
osc.indices.create(index="poma", body={"settings": {"index.knn": True}, # knn_vector fields need index.knn=true, otherwise the mapping is rejected (and ignore=400 hides it)
"mappings": {"properties": {
"embedding": {"type": "knn_vector", "dimension": embedder.dims,
"method": {"name": "hnsw", "space_type": "cosinesimil", "engine": "faiss"}},
"file_id": {"type": "keyword"},
"chunkset_index": {"type": "integer"},
}}}, ignore=400)
# 2. Content-free ingest — OpenSearch holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
osc.index(index="poma", id=r.id, body={
"embedding": embedder.embed([r.text])[0],
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
})
# 3. Retrieval — OpenSearch's k-NN plugin query, then assemble.
qv = embedder.embed(["What are the early termination conditions?"])[0]
osc.indices.refresh(index="poma") # make the just-indexed docs searchable before querying
res = osc.search(index="poma", body={"query": {"knn": {"embedding": {
"vector": qv, "k": 10}}}})
context = assemble(res, volume=vol) # -> [{"file_id", "content"}, ...]records_from_archive reads the .poma archive PrimeCut just wrote and returns one Record per chunkset: .id (deterministic chunkset_uuid(file_id, chunkset_index)), .text (the to_embed string used to build the embedding), and .payload (file_id, chunkset_index, chunks). The same id scheme means re-ingesting a document overwrites the same OpenSearch documents rather than duplicating them. On the reference legal-document benchmark this discipline answers with 337 tokens of retrieved context instead of 1,542 for a recursive-splitter baseline — methodology here.
Metadata mapping: POMA fields → OpenSearch primitives
| POMA field | Where it lives | OpenSearch role |
|---|---|---|
to_embed | embedded, written as the embedding field | knn_vector field for the k-NN plugin's ranking |
file_id | document field (keyword) | filterable via bool.filter, returned by default |
chunkset_index | document field (integer) + volume key | forms the deterministic id; also returned by default |
page, depth, chunks (from Marker's Page blocks and block tree) | volume only (record.payload) | reconstructed into cheatsheet content, never indexed as an OpenSearch field |
| chunkset lineage | volume document | deduplicated and merged into one cheatsheet by assemble() |
Frequently asked questions
How do I get Marker output into OpenSearch for RAG?
Run Marker with the JSON renderer (marker_single --output_format json), hand the saved result to PrimeCut().ingest(), and keep the .poma archive. Turn each chunkset into a Record with records_from_archive, embed record.text, and osc.index() a content-free document (embedding, file_id, chunkset_index) per record. Retrieve with the k-NN plugin's knn query and pass the response to assemble().
Why not just split Marker's markdown and index it into OpenSearch directly?
Marker's markdown concatenates every page into one string with no boundaries, and leaves image bytes in a side-channel dict as dangling refs. A generic splitter also cuts the JSON renderer's full HTML tables mid-row — and if you add OpenSearch's hybrid query type on top without a normalization search pipeline, unbounded lexical scores from that leftover table markup can outweigh a properly normalized vector score.
What OpenSearch mapping fields should Marker chunksets carry?
embedding as knn_vector (dimension matching your embedder, HNSW via faiss/nmslib), plus file_id (keyword) and chunkset_index (integer) as routing fields — the content-free contract vektoria writes by default.
Do Marker's tables and images survive the trip to OpenSearch?
Yes, upstream of OpenSearch. PrimeCut splices Marker's side-channel images dict back into its references and describes each figure, and keeps full HTML tables intact through the chunk layer. OpenSearch itself never stores images or tables under this pattern — only the vector and two routing fields — and the reconstructed content comes back from the volume via assemble().
Does OpenSearch's hybrid query pipeline need extra configuration for Marker-sourced chunksets?
Only if you add lexical search on top of the plain knn query shown here. OpenSearch's hybrid query type needs a search pipeline to normalize BM25 and vector scores — a distinct mechanism from Elasticsearch's RRF rank — and skipping it lets unnormalized noise from any leftover table markup outrank a properly-scaled vector hit. The plain knn retrieval above needs no such step and returns file_id/chunkset_index by default.
Related recipes
Same parser, different store: Marker → Qdrant · Marker → Weaviate · Marker → Chroma
Foundations: The Optimal Chunker for Marker · OpenSearch Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide