Skip to content

Marker to Elasticsearch: Chunking for RAG ​

The short answer: Marker gives you fast, local PDF parsing with a real Document block tree — per-page blocks, explicit types, and full HTML tables with spans intact, when you run its JSON renderer. Elasticsearch gives you mature hybrid search, with dense_vector k-NN and native BM25 in a single request. Wired together naively, Marker's page boundaries and table structure are gone before a document is indexed, and Elasticsearch ranks the resulting broken fragments with full confidence. The missing link is POMA: PrimeCut().ingest() turns the saved Marker JSON into hierarchy-preserving chunksets, and poma.vektoria indexes them as content-free documents — embedding, file_id, chunkset_index — while the actual chunkset text lives on a volume. Retrieval runs Elasticsearch's own knn query, then assemble() reconstructs prompt-ready cheatsheets.

What each end of the pipeline actually provides ​

Marker (by Datalab / VikParuchuri) runs on your own hardware and turns PDFs into clean structured output. With the JSON renderer (--output_format json) you get a Document block tree — Page children, per-block HTML, and full <table> elements with row and column spans intact. What it doesn't provide: cross-page hierarchy, retrieval units, or inline images — figure bytes live in a side-channel images dict keyed by name, with only ![](name) references left in the text. Details: The Optimal Chunker for Marker.

Elasticsearch provides dense_vector fields with HNSW-backed approximate k-NN, native BM25 on any analyzed text field, a single request combining knn with a bool/match query, and rank (reciprocal rank fusion) to merge them server-side. What it doesn't provide: any opinion about what a document should represent — a mapping whose fields hold a broken table fragment or a dangling image reference gets ranked exactly as confidently as a clean one. Details: Elasticsearch Chunking Strategy for RAG.

The naive wiring, and where it breaks ​

The common recipe — concatenate Marker's per-page markdown into one string, RecursiveCharacterTextSplitter, embed, es.index() the raw chunk text — breaks in ways specific to this pair:

  • Page boundaries disappear before a single document is indexed. Marker's markdown output alone has no page delimiters, so a flatten-then-split pipeline can't populate a page field to filter or cite by — only the JSON renderer's Page blocks carry that boundary, and only if the ingest path reads them.
  • Tables get cut mid-row. The JSON renderer's tables arrive as full <table> elements with colspans — real structure Marker recovered. A character splitter run over the flattened markdown routinely slices a table in half; Elasticsearch then indexes and ranks half a <tr> as if it were complete.
  • Every figure vanishes. Marker's images live in the side-channel images dict; the markdown carries only ![](name) references, which a markdown-only pipeline embeds as literal, unmatched text — the figure itself was never indexed.
  • Overlap inflates the index. Splitter overlap duplicates every boundary span into both the HNSW graph and the BM25 inverted index, crowding knn results with near-duplicates of the same passage.
  • Heading levels are discarded, so a retrieved fragment arrives without lineage and the LLM answers out of context — the failure Marker's parsing quality can't fix.

The pipeline, end to end ​

bash
pip install elasticsearch

# Your existing Marker run — unchanged. The JSON renderer keeps the
# Document block tree that POMA auto-detects, tables and pages included.
marker_single contract.pdf --output_format json --output_dir out/
python
from poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from elasticsearch import Elasticsearch

# 1. Chunk the raw Marker JSON and keep the .poma archive — the volume's source of truth.
client = PrimeCut()  # reads POMA_API_KEY
client.ingest("out/contract/contract.json", download_dir="archives", filename="contract.poma")

embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
es = Elasticsearch("http://localhost:9200")  # local dev cluster (xpack.security.enabled=false); a secured cluster needs https + basic_auth/api_key + ca_certs

es.indices.create(index="poma", mappings={"properties": {
    "embedding": {"type": "dense_vector", "dims": embedder.dims, "index": True, "similarity": "cosine"},
    "file_id": {"type": "keyword"},
    "chunkset_index": {"type": "integer"},
}}, ignore=400)

# 2. Content-free ingest — Elasticsearch holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma")  # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
    es.index(index="poma", id=r.id, document={
        "embedding": embedder.embed([r.text])[0],
        "file_id": r.payload["file_id"],
        "chunkset_index": r.payload["chunkset_index"],
    })

# 3. Retrieval — Elasticsearch's normal knn query, then assemble.
qv = embedder.embed(["What are the early termination conditions?"])[0]
es.indices.refresh(index="poma")  # make the just-indexed docs searchable before querying
res = es.search(index="poma", knn={"field": "embedding", "query_vector": qv,
                                    "k": 10, "num_candidates": 100})
context = assemble(res.body, volume=vol)  # elasticsearch-py 8+ wraps the response (ObjectApiResponse) — pass the dict body  # -> [{"file_id", "content"}, ...]

records_from_archive reads the .poma archive PrimeCut just wrote and returns one Record per chunkset: .id (deterministic chunkset_uuid(file_id, chunkset_index)), .text (the to_embed string used to build the embedding), and .payload (file_id, chunkset_index, chunks). The same id scheme means re-ingesting a document overwrites the same Elasticsearch documents rather than duplicating them. On the reference legal-document benchmark this discipline answers with 337 tokens of retrieved context instead of 1,542 for a recursive-splitter baseline — methodology here.

Metadata mapping: POMA fields → Elasticsearch primitives ​

POMA fieldWhere it livesElasticsearch role
to_embedembedded, written as the embedding fielddense_vector field for knn ranking
file_iddocument field (keyword)filterable via bool.filter, returned in _source by default
chunkset_indexdocument field (integer) + volume keyforms the deterministic id; also returned in _source by default
page, depth, chunks (from Marker's Page blocks and block tree)volume only (record.payload)reconstructed into cheatsheet content, never indexed as an Elasticsearch field
chunkset lineagevolume documentdeduplicated and merged into one cheatsheet by assemble()

Frequently asked questions ​

How do I get Marker output into Elasticsearch for RAG? ​

Run Marker with the JSON renderer (marker_single --output_format json), hand the saved result to PrimeCut().ingest(), and keep the .poma archive. Turn each chunkset into a Record with records_from_archive, embed record.text, and es.index() a content-free document (embedding, file_id, chunkset_index) per record. Retrieve with es.search(knn=...) and pass the response to assemble().

Why not just split Marker's markdown and index it into Elasticsearch directly? ​

Marker's markdown concatenates every page into one string with no boundaries, and leaves image bytes in a side-channel dict as dangling refs. A generic splitter run over that flat string also cuts the JSON renderer's full HTML tables mid-row, and Elasticsearch ranks the resulting broken fragment with full confidence — nothing in the mapping flags it as truncated.

What Elasticsearch mapping fields should Marker chunksets carry? ​

embedding as dense_vector (dims matching your embedder, similarity: cosine), plus file_id (keyword) and chunkset_index (integer) as routing fields — the content-free contract vektoria writes by default.

Do Marker's tables and images survive the trip to Elasticsearch? ​

Yes, upstream of Elasticsearch. PrimeCut splices Marker's side-channel images dict back into its references and describes each figure, and keeps full HTML tables intact through the chunk layer. Elasticsearch itself never stores images or tables under this pattern — only the vector and two routing fields — and the reconstructed content comes back from the volume via assemble().

Does Elasticsearch retrieval need extra fields for POMA's assemble() to work? ​

No extra ask needed — Elasticsearch returns the full _source by default on a knn search, so file_id and chunkset_index come back on every hit without an explicit projection.

Same parser, different store: Marker → Qdrant · Marker → Pinecone · Marker → pgvector

Foundations: The Optimal Chunker for Marker · Elasticsearch Chunking Strategy for RAG · All pipeline recipes · RAG chunking guide