Elasticsearch Chunking Strategy: semantic_text and Chunksets
The short answer: Elasticsearch chunking is built in — a semantic_text field splits and embeds text inside the inference endpoint, and chunking_settings picks the strategy (sentence, word, recursive, none). What it cannot do is decide what a chunk carries with it. The optimal chunks for Elasticsearch are chunksets — self-explanatory units that carry their full heading lineage — indexed as documents with a dense_vector field for the embedding and file_id/page/depth as filterable keyword/integer fields, with the analyzed text field enabling native BM25 alongside k-NN. Elasticsearch's ranking is only as good as what you indexed; POMA's PrimeCut emits exactly the fields this mapping needs, and its vektoria package keeps the index content-free so retrieval assembles clean context from a volume instead of a bloated _source.
Elasticsearch gives you mature hybrid search — over whatever you index
Elasticsearch's vector support sits on top of two decades of full-text search maturity:
dense_vectorfields — HNSW-backed approximate k-NN,similarity: cosine(ordot_product,l2_norm) declared per field in the mapping.- Native BM25 — every analyzed text field participates in relevance scoring without a second system.
- One request, both signals — a
knnclause and abool/matchquery combine in the same search body;rank(reciprocal rank fusion) merges them server-side. - Mature filtering —
bool.filteronkeyword/numericfields runs at index speed, not a post-filter scan.
None of this decides what a document should represent. Elasticsearch ranks whatever vector and text you indexed. A mapping whose text field holds a context-free character-count fragment will rank context-free fragments — correctly, and fast.
How Elasticsearch chunking works: semantic_text and chunking_settings
Elasticsearch splits text for you. Map a field as semantic_text and every value you index is sent to an inference endpoint, split into chunks there, embedded, and stored back on the document. The embeddings are internal and excluded from the response by default.
PUT semantic-embeddings
{
"mappings": {
"properties": {
"content": {
"type": "semantic_text",
"inference_id": ".elser-2-elasticsearch"
}
}
}
}Splitting is controlled by chunking_settings, which you set on the inference endpoint (PUT _inference/...) or directly on the field mapping:
"content": {
"type": "semantic_text",
"inference_id": "my-inference-endpoint",
"chunking_settings": { "strategy": "word", "max_chunk_size": 120, "overlap": 40 }
}The settings, with the values the API reference documents:
| Setting | Values and default | Applies to |
|---|---|---|
strategy | sentence (default), word, recursive, none | all |
max_chunk_size | maximum chunk size in words, default 250; minimum 20 for sentence, 10 for word | all but none |
overlap | overlapping words, default 100, at most half of max_chunk_size | word |
sentence_overlap | 1 or 0, default 1 | sentence |
separators | list of strings or regex patterns, tried in order | recursive |
separator_group | markdown or plaintext (prebuilt separator lists) | recursive |
recursive requires max_chunk_size plus either separators or separator_group. Query the field with a match query, or with the : operator in ES|QL.
Elasticsearch semantic chunking: what sentence and recursive actually do
"Semantic chunking" usually means: split on meaning, not on a character count. Elasticsearch has two structure-aware strategies that get partway there.
sentencepacks whole sentences untilmax_chunk_sizewords is reached, and repeats the last sentence of a chunk into the next one whensentence_overlapis1. Boundaries land on punctuation, never mid-word.recursivetakes an orderedseparatorslist and tries each split point in turn, then recombines the pieces into chunks that fitmax_chunk_size.separator_group: markdownsupplies a prebuilt list, so headings and list items become split candidates before paragraphs and sentences do.
Neither strategy splits on embedding similarity. Both are boundary heuristics, and a good boundary is not the same thing as sufficient context. separator_group: markdown will split at an ## heading; it does not attach that heading to the passages underneath. The word "Termination" stays in the heading's chunk. The clause that defines termination is embedded without it.
How do I use ingest pipelines in Elasticsearch to recursively chunk documents?
Two answers, and the first one is usually the one you want.
Use the recursive strategy, not a pipeline. Recursive splitting lives in chunking_settings, not in the ingest pipeline API. Elasticsearch has no chunking processor — there is no equivalent of OpenSearch's text_chunking processor. Configure it once on the inference endpoint and every semantic_text field pointing at that endpoint inherits it:
PUT _inference/text_embedding/my-recursive-endpoint
{
"service": "elasticsearch",
"service_settings": {
"num_allocations": 1,
"num_threads": 1,
"model_id": ".multilingual-e5-small"
},
"chunking_settings": {
"strategy": "recursive",
"max_chunk_size": 300,
"separator_group": "markdown"
}
}Swap separator_group for an explicit separators list when your documents have their own structure — a numbered-clause regex for contracts, for example. The list is tried in order, coarsest separator first.
What the ingest pipeline is still for. Ingest processors run before inference, so a pipeline is the right place to normalize text, strip boilerplate headers, or route a source field into the semantic_text field with set, script, split and foreach. What it cannot do is produce the passages. If you want the passages under your own control, split outside Elasticsearch and index one document per passage — which is the next section.
What Elasticsearch's built-in chunking leaves out
chunking_settings decides where the cut falls. Nothing in it decides what the cut carries with it, and that is the part that decides whether a retrieved hit is answerable.
Two consequences show up in production:
- A chunk carries no document path. Chunk 7 of a 40-page agreement is a span of words. It does not know it sits under
Master Services Agreement → Termination clauses → Early termination. Retrieval returns it, the model reads "Either party may…", and nothing in the prompt says which party or which agreement. - You cannot filter on a field you never indexed.
semantic_textchunks are internal subfields of one document, not documents. There is no place to hangdepth: 3orpage: 17on chunk 7, sobool.filtercannot scope a query to one hierarchy level or one page range. Filtering happens at document granularity or not at all.
Both are fixed the same way: make the unit of retrieval a unit that already carries its context, and give it real fields.
Two ways to index chunksets in Elasticsearch
Option 1 — chunksets as documents with a dense_vector field. One document per chunkset, to_embed embedded with your model, hierarchy as keyword/integer fields. That is the mapping in the pipeline above, and it is the one to reach for when you already run an embedder. page and depth become bool.filter clauses, and text stays analyzed for BM25.
Option 2 — semantic_text with strategy: none. Keep Elasticsearch in charge of inference, and turn its splitter off, because PrimeCut has already split the document:
PUT chunksets
{
"mappings": {
"properties": {
"content": {
"type": "semantic_text",
"inference_id": "my-inference-endpoint",
"chunking_settings": { "strategy": "none" }
},
"file_id": { "type": "keyword" },
"chunkset_index": { "type": "integer" },
"page": { "type": "integer" },
"depth": { "type": "integer" }
}
}
}Index chunkset.to_embed as content and the hierarchy alongside it. strategy: none embeds the field value as a single chunk, which is correct here: a chunkset is already sized by document structure, and re-splitting it would throw away the lineage it was built to carry. One caveat — the model's own input window still applies, so check the endpoint's limit against your longest chunkset.
What "optimal chunks" means for Elasticsearch, concretely
- One document per chunkset, self-explanatory alone. A chunkset is a root-to-leaf path —
Master Services Agreement → Termination clauses → Early termination → "Either party may…". See POMA chunksets. - Hierarchy as filterable fields.
file_id(keyword),page(integer),depth(integer) turnbool.filterinto document-aware scoping — one contract, one page range, one hierarchy level. - BM25-eligible text, always. Keep the chunkset's
to_embedtext in an analyzed field so hybrid search has both a vector and a lexical signal to fuse. - No overlap. Overlap duplicates spans into both the HNSW graph and the inverted index. Chunksets need no overlap.
- Content-free index. Store
{file_id, chunkset_index}in the index and the actual content on a volume — smaller index, faster merges, and citations reconstructed at retrieval time.
The pipeline: PrimeCut to Elasticsearch
pip install elasticsearchfrom poma import PrimeCut
from poma.vektoria import assemble, records_from_archive, Volume
from poma.embeddings import get_embedder
from elasticsearch import Elasticsearch
# 1. Chunk any document and keep the .poma archive — the volume's source of truth.
client = PrimeCut() # reads POMA_API_KEY
client.ingest("contract.pdf", download_dir="archives", filename="contract.poma")
embedder = get_embedder("local:BAAI/bge-small-en-v1.5")
vol = Volume("s3://your-bucket/poma")
es = Elasticsearch("http://localhost:9200") # local dev cluster (xpack.security.enabled=false); a secured cluster needs https + basic_auth/api_key + ca_certs
es.indices.create(index="poma", mappings={"properties": {
"embedding": {"type": "dense_vector", "dims": embedder.dims, "index": True, "similarity": "cosine"},
"file_id": {"type": "keyword"},
"chunkset_index": {"type": "integer"},
"text": {"type": "text"},
}}, ignore=400)
# 2. Content-free ingest — vector index holds only routing metadata; content lives on the volume.
records = records_from_archive("archives/contract.poma") # -> list[Record] (id, text, payload)
# assemble() reads the FULL document (chunks + chunksets) from the volume — write it once,
# not one {"chunks": <indices>, "text"} stub per record (which also overwrote itself each turn).
from poma.utils import unpack_poma_archive
_data = unpack_poma_archive(poma_archive_path="archives/contract.poma")
file_id = records[0].payload["file_id"]
vol.write_doc(file_id, {"file_id": file_id, "chunks": _data["chunks"], "chunksets": _data["chunksets"]})
for r in records:
es.index(index="poma", id=r.id, document={
"embedding": embedder.embed([r.text])[0],
"file_id": r.payload["file_id"],
"chunkset_index": r.payload["chunkset_index"],
"text": r.text,
})
# 3. Hybrid k-NN + BM25 retrieval, then assemble prompt-ready context.
qv = embedder.embed(["early termination conditions"])[0]
es.indices.refresh(index="poma") # make the just-indexed docs searchable before querying
res = es.search(index="poma", knn={"field": "embedding", "query_vector": qv,
"k": 10, "num_candidates": 100})
context = assemble(res.body, volume=vol) # elasticsearch-py 8+ wraps the response (ObjectApiResponse) — pass the dict body # -> [{"file_id", "content"}, ...]Retrieved chunksets share ancestor lineage across a document — assemble() deduplicates that lineage into one prompt-ready cheatsheet, the same discipline that answered one reference legal document with 337 tokens instead of 1,542 for a recursive-splitter baseline. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.
Chunk shapes in Elasticsearch, compared
| Property | Fixed 512-token chunks + overlap | Semantic chunks | POMA chunksets |
|---|---|---|---|
| Document is self-explanatory alone | ✗ | Sometimes | ✓ (by construction) |
| Hierarchy as filterable fields | Manual | Manual | ✓ (file_id, page, depth) |
| Redundant HNSW/BM25 entries | Many (overlap) | Few | None |
| Content-free index + volume | DIY | DIY | ✓ (vektoria assemble/Volume) |
| Metadata returned on hits | Default | Default | Default — no extra flag needed |
Frequently asked questions
How does chunking work in Elasticsearch?
Elasticsearch chunks text inside the inference endpoint. A semantic_text field sends each indexed value to the endpoint, where it is split, embedded, and stored back on the document. chunking_settings — on the endpoint or on the field mapping — controls it: strategy is sentence (default), word, recursive or none; max_chunk_size is in words, default 250; overlap (default 100) applies to word; sentence_overlap (1 or 0, default 1) applies to sentence. There is no chunking processor in the ingest pipeline API.
What is the best Elasticsearch chunking strategy for RAG?
The one that produces chunks a model can read alone. Of the built-ins, recursive with separator_group: markdown gets closest, because it splits on structure first. It still does not attach a heading lineage to the passages under a heading. Chunksets do: index one document per chunkset with a dense_vector embedding plus file_id/page/depth, or keep semantic_text with strategy: none.
Does Elasticsearch support semantic chunking?
It supports structure-aware chunking, which is what most people mean. sentence packs whole sentences up to max_chunk_size and repeats the last one when sentence_overlap is 1; recursive tries an ordered separators list (or separator_group: markdown/plaintext) and recombines pieces that fit. Neither splits on embedding similarity, and neither carries a heading into the passages below it.
How do I use ingest pipelines in Elasticsearch to recursively chunk documents?
Recursive chunking is not an ingest processor — Elasticsearch has no equivalent of OpenSearch's text_chunking. Set chunking_settings to strategy: recursive with a max_chunk_size and either separators or separator_group (markdown or plaintext), on the inference endpoint or the semantic_text mapping. The pipeline's job is what runs before inference: set, script, split, foreach for normalizing and routing. To control the passages yourself, split outside Elasticsearch and index one document per passage.
What is the optimal chunk size for Elasticsearch?
There's no universal token count — chunksets are self-explanatory at any size, which is why they beat fixed 512-token windows. Start at 512 tokens if you must fix a size, but the size knob cannot fix missing context.
How do I map chunksets into an Elasticsearch index?
embedding as dense_vector (dims matching your embedder, similarity: cosine), file_id/page as keyword/integer, text as an analyzed field for BM25 — one document per chunkset.
Does Elasticsearch support hybrid vector and keyword search?
Yes — a single request combines knn with bool/match, and rank (RRF) fuses the two ranked lists server-side.
How does POMA retrieve context from Elasticsearch results?
vektoria keeps the index content-free (vector + file_id/chunkset_index only); content lives on a volume. assemble(results, volume=VOL) auto-detects Elasticsearch's result shape and returns deduplicated cheatsheets — no extra metadata flag needed.
How does chunk overlap affect an Elasticsearch index?
It duplicates spans into both the HNSW graph and the BM25 inverted index — more memory, top-k crowded with near-duplicates. Chunksets need no overlap.
Feed Elasticsearch from the parser you already run
- AWS Textract → Elasticsearch
- Azure Document Intelligence → Elasticsearch
- Docling → Elasticsearch
- LlamaParse → Elasticsearch
- Marker → Elasticsearch
- Mistral OCR → Elasticsearch
- PaddleOCR-VL → Elasticsearch
- Unstructured.io → Elasticsearch
Running a different store? The same chunk design applies: Azure AI Search · Chroma · FAISS · LanceDB · Milvus · MongoDB Atlas · OpenSearch · pgvector · Pinecone · Qdrant · Redis · Turbopuffer · Vespa · Weaviate — or browse all pipeline recipes.
Fundamentals: RAG chunking guide · RAG architecture guide · POMA chunksets.