Source: http://www.poma-ai.com/docs/optimal-chunker-llamaindex

# LlamaIndex Chunking for RAG: Node Parsers and Chunksets

<ByAuthor />

LlamaIndex chunking happens in one place: the node parser. A node parser takes `Document` objects and returns `Node` objects, and everything else in the framework, indexes, retrievers, query engines, works on those nodes. Which node parser you choose decides where the cuts fall, what each retrieved unit knows about itself, and how much context your LLM has to be given at answer time.

This page covers the built-in options with their real API names, shows `HierarchicalNodeParser` with `AutoMergingRetriever` in full, and then shows a node parser whose units are document-structure paths instead of size tiers. Your index and query engine code does not change.

## Quick start: chunkset nodes in LlamaIndex

`PomaChunksetNodeParser` is a `NodeParser` subclass. Swap it in where your `SentenceSplitter` is today.

```bash
pip install 'poma[llamaindex]'
```

```python
from llama_index.core import VectorStoreIndex
from llama_index.core.query_engine import RetrieverQueryEngine
from poma import PrimeCut
from poma.integrations.llamaindex import (
    PomaFileReader,
    PomaChunksetNodeParser,
    PomaCheatsheetRetrieverLI,
)

# 1. Load files. PDF, DOCX, HTML, Markdown, CSV, XLSX, images and more.
documents = PomaFileReader().load_data("./contracts")

# 2. Chunk. One node per chunkset: a root-to-leaf path through the heading tree.
parser = PomaChunksetNodeParser(client=PrimeCut())  # reads POMA_API_KEY
nodes = parser.get_nodes_from_documents(documents, show_progress=True)

# 3. Index and query. Unchanged LlamaIndex code from here on.
index = VectorStoreIndex(nodes)
retriever = PomaCheatsheetRetrieverLI(index.as_retriever(similarity_top_k=4))
query_engine = RetrieverQueryEngine.from_args(retriever)
print(str(query_engine.query("What are the termination conditions?")))
```

`PomaFileReader` writes `source_path` and a content-hash `doc_id` into each `Document`'s metadata; the node parser needs `source_path` to be a real file, because PrimeCut does the parsing and the chunking server-side. Details of the three helpers are in the [LlamaIndex integration reference](/sdk/integrations/llamaindex).

## Node parser vs text splitter

The two terms are used interchangeably in tutorials and they are not the same thing.

A **node parser** is the general abstraction. The LlamaIndex docs describe it as a "simple abstraction that take a list of documents, and chunk them into `Node` objects, such that each node is a specific chunk of the parent document." It also sets node relationships, `NodeRelationship.SOURCE`, `PREVIOUS`, `NEXT`, `PARENT` and `CHILD`, and propagates document metadata: "all of it's attributes are inherited to the children nodes (i.e. `metadata`, text and metadata templates, etc.)."

A **text splitter** is a node parser whose rule is only about splitting a string: `SentenceSplitter`, `TokenTextSplitter`, `CodeSplitter`. `MarkdownNodeParser`, `JSONNodeParser` and `HierarchicalNodeParser` are node parsers but not text splitters, because they read structure rather than counting characters.

Any node parser can be used three ways:

```python
# Standalone
nodes = node_parser.get_nodes_from_documents(documents)

# As a transformation in an ingestion pipeline
from llama_index.core.ingestion import IngestionPipeline
pipeline = IngestionPipeline(transformations=[node_parser])
nodes = pipeline.run(documents=documents)

# Globally
from llama_index.core import Settings
Settings.text_splitter = node_parser
```

`Settings.chunk_size` and `Settings.chunk_overlap` are shortcuts that adjust the default splitter without replacing it.

## LlamaIndex chunking strategies, and what each one cuts on

Every built-in parser is a different answer to one question: where does a chunk end?

**Fixed size.** `SentenceSplitter(chunk_size=1024, chunk_overlap=20)` is the default. It respects sentence boundaries while filling a token budget. `TokenTextSplitter(chunk_size=1024, chunk_overlap=20, separator=" ")` does not, and cuts on raw tokens.

**Recursive.** In LlamaIndex, "recursive chunking" is what `SentenceSplitter` already does: it splits while respecting sentence boundaries rather than cutting on a raw count. It is the same idea as LangChain's `RecursiveCharacterTextSplitter`, so if you arrived from a LangChain tutorial, `SentenceSplitter` is the parser you were looking for.

**Semantic.** `SemanticSplitterNodeParser(buffer_size=1, breakpoint_percentile_threshold=95, embed_model=embed_model)` embeds groups of sentences and cuts where the distance between neighbouring groups crosses the percentile threshold. It needs an embedding model at index time and costs one embedding call per sentence group.

**Format-aware.** `MarkdownNodeParser()` cuts on headers, `JSONNodeParser()` on JSON structure, `CodeSplitter(language="python", chunk_lines=40, chunk_lines_overlap=15, max_chars=1500)` on syntax.

**Window.** `SentenceWindowNodeParser.from_defaults(window_size=3, window_metadata_key="window", original_text_metadata_key="original_sentence")` embeds one sentence and stores its neighbours in metadata, so retrieval is precise and the LLM still gets surrounding text. It buys local context, not global context.

**Hierarchical.** `HierarchicalNodeParser`, covered next, is the only built-in that produces nodes at more than one granularity.

The [full taxonomy with the trade-offs](/rag-chunking-strategies-text-splitters) covers these across frameworks.

## Hierarchical chunking with AutoMergingRetriever

This is the closest built-in LlamaIndex has to hierarchy-aware retrieval, and it is worth understanding precisely before reaching for anything else.

```python
from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.core.node_parser import (
    HierarchicalNodeParser,
    get_leaf_nodes,
)
from llama_index.core.retrievers import AutoMergingRetriever
from llama_index.core.storage.docstore import SimpleDocumentStore

node_parser = HierarchicalNodeParser.from_defaults(chunk_sizes=[2048, 512, 128])
nodes = node_parser.get_nodes_from_documents(documents)
leaf_nodes = get_leaf_nodes(nodes)

docstore = SimpleDocumentStore()
docstore.add_documents(nodes)               # every level goes in the docstore
storage_context = StorageContext.from_defaults(docstore=docstore)

base_index = VectorStoreIndex(leaf_nodes, storage_context=storage_context)
base_retriever = base_index.as_retriever(similarity_top_k=6)
retriever = AutoMergingRetriever(base_retriever, storage_context, verbose=True)
```

The document is split three times, at 2048, 512 and 128 tokens. `chunk_overlap` defaults to 20 at every level. Nodes are linked with `PARENT` and `CHILD`, which is exactly what `get_leaf_nodes` reads: it returns nodes with no `NodeRelationship.CHILD`. Only leaves are embedded. At query time `AutoMergingRetriever` counts how many children of a parent were retrieved and, past a threshold, swaps them for the parent.

It works. It also has two properties that matter:

- **The hierarchy is a size hierarchy.** 2048, 512 and 128 tokens are token budgets, not chapters, sections and paragraphs. A 2048-token parent starts and ends wherever the counter said, so it may open mid-sentence in one clause and close inside the next.
- **A merged parent is still flat text.** When the retriever hands you the 2048-token parent, that text does not say which chapter it came from. It is a bigger fragment, not a located one.

Both are inherent to splitting text by length. Neither is a defect in the implementation.

## Chunkset nodes: cut points from the document, not the token counter

PrimeCut ([PomaChunksetNodeParser](/sdk/integrations/llamaindex)) is a full ingestion and chunking engine: it parses the source file, including scanned PDFs, rebuilds the heading tree across pages, keeps tables whole, and emits **chunks** and **chunksets**. A [chunkset](/learn/chunking/chunksets) is a root-to-leaf path through that tree, so a sentence arrives with its chapter, section and paragraph attached. There is no overlap parameter, because nothing was cut in the middle of an idea.

In LlamaIndex terms, one chunkset becomes one `TextNode`:

```python
node = nodes[0]
node.text                        # chunkset.to_embed: content with its heading path
node.metadata["doc_id"]          # content hash from PomaFileReader
node.metadata["chunkset_index"]  # position of the chunkset in the document
node.metadata["source_path"]     # original file
node.metadata["chunkset"]        # the raw chunkset dict
node.metadata["chunks"]          # its member chunks: chunk_index, content, depth, file_id
node.excluded_embed_metadata_keys  # every metadata key: only the text is embedded
```

Because it is a `NodeParser`, it drops into an ingestion pipeline unchanged:

```python
from llama_index.core.ingestion import IngestionPipeline

pipeline = IngestionPipeline(
    transformations=[PomaChunksetNodeParser(client=PrimeCut())]
)
nodes = pipeline.run(documents=PomaFileReader().load_data("./contracts"))
```

If you already run a parser you like, LlamaParse included, PrimeCut can take that parser's saved result JSON instead of the original file and run only the chunking stages: see [the optimal chunker for LlamaParse](/optimal-chunker-llamaparse).

On a legal document in our reference set, the same question was answered from 337 tokens of retrieved context with chunksets versus 1,542 with a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the [ingestion guide](/document-ingestion-chunking-rag) has the broader numbers.

## LlamaIndex node parsers compared

| Node parser | Boundary rule | Unit carries its context? | Embedding model at index time? | Use it when |
| --- | --- | --- | --- | --- |
| `TokenTextSplitter` | Token count | No | No | You need a hard token ceiling and nothing else |
| `SentenceSplitter` | Sentence boundary inside a token budget | No | No | Default for prose; the baseline to beat |
| `SentenceWindowNodeParser` | One sentence, neighbours in metadata | Local only | No | Precise retrieval, small surrounding window |
| `SemanticSplitterNodeParser` | Embedding distance percentile | No | Yes | Topic shifts matter and cost is acceptable |
| `MarkdownNodeParser` | Markdown headers | No | No | Clean Markdown, headings already correct |
| `JSONNodeParser` | JSON structure | No | No | Structured JSON records |
| `CodeSplitter` | Language syntax | No | No | Source code |
| `HierarchicalNodeParser` + `AutoMergingRetriever` | Three token sizes, linked PARENT/CHILD | Larger, not located | No | Small units to search, bigger ones to read |
| `PomaChunksetNodeParser` | Document heading tree | Yes: root-to-leaf path | No | Real documents with structure, tables, pages |

"Carries its context" is the column that decides answer quality. Everything above it is about where the cut lands; that column is about what the retrieved unit knows about itself.

## Frequently asked questions

### How does LlamaIndex chunking work?

Chunking in LlamaIndex is done by a node parser: a class that takes a list of Document objects and returns Node objects. Call get_nodes_from_documents(documents) on it, put it in IngestionPipeline(transformations=[...]), or set it globally with Settings.text_splitter. SentenceSplitter(chunk_size=1024, chunk_overlap=20) is the default. Which node parser you pick decides where the cuts fall and how much context each retrieved node carries.

### What is the difference between a node parser and a text splitter in LlamaIndex?

Text splitters are a subset of node parsers. A node parser is the general abstraction that turns Documents into Nodes and sets relationships (SOURCE, PREVIOUS, NEXT, PARENT, CHILD) and inherited metadata. A text splitter is a node parser whose rule is purely about splitting a text string, such as SentenceSplitter or TokenTextSplitter. Parsers like MarkdownNodeParser, JSONNodeParser and HierarchicalNodeParser are node parsers but not plain text splitters, because they read structure rather than only character or token counts.

### LlamaIndex chunk size and overlap: what should I use?

SentenceSplitter defaults to chunk_size=1024 and chunk_overlap=20, and you can set them globally with Settings.chunk_size and Settings.chunk_overlap. There is no size that is right for every corpus, which is the real problem: overlap exists to repair context that the cut destroyed. A structure-aware unit needs no overlap at all, because it is bounded by the document's own headings rather than by a token budget.

### LlamaIndex semantic chunking: how does it work?

SemanticSplitterNodeParser(buffer_size=1, breakpoint_percentile_threshold=95, embed_model=embed_model) embeds groups of sentences and cuts where the embedding distance between neighbouring groups exceeds the given percentile. buffer_size is the number of sentences grouped together when evaluating semantic similarity. It costs embedding calls at index time and it finds topic shifts, but it never learns that a paragraph belongs to a chapter, because it only sees local similarity.

### LlamaIndex hierarchical chunking: how does it work?

HierarchicalNodeParser.from_defaults(chunk_sizes=[2048, 512, 128]) splits the same document three times at three sizes and links the results with PARENT and CHILD relationships. You index only get_leaf_nodes(nodes), put every node in a docstore, and wrap the base retriever in AutoMergingRetriever, which replaces a set of retrieved leaves with their shared parent when enough of them hit. The hierarchy is a size hierarchy: 2048, 512 and 128 tokens are not chapters, sections and paragraphs.

### Does LlamaIndex have agentic chunking?

There is no built-in agentic node parser. What people call agentic chunking in LlamaIndex is usually assembled from an LLM step in an IngestionPipeline, for example a metadata extractor that writes a title or summary onto nodes a splitter already produced. It is an LLM call per chunk on every ingest, it is not reproducible between runs, and the boundaries still come from the underlying splitter.

### Can I use PrimeCut chunksets as LlamaIndex nodes?

Yes. PomaChunksetNodeParser is a LlamaIndex NodeParser subclass, so it works anywhere a node parser works: get_nodes_from_documents, IngestionPipeline transformations, or an index construction call. Each returned TextNode carries one chunkset as its text, with doc_id, chunkset_index, source_path, the raw chunkset and its member chunks in metadata, and all metadata keys added to excluded_embed_metadata_keys so only the content is embedded.

### Do I have to change my LlamaIndex index or query engine code?

No. PomaChunksetNodeParser returns ordinary TextNode objects, so VectorStoreIndex, your vector store integration, as_retriever and as_query_engine all stay the same. The only optional extra is PomaCheatsheetRetrieverLI, which wraps an existing retriever and merges hits from the same document into one cheatsheet node before the LLM sees them.

## Continue reading

- [LlamaIndex integration reference](/sdk/integrations/llamaindex) — the three helpers, installed and configured
- [RAG chunking strategies and text splitters](/rag-chunking-strategies-text-splitters) — the full taxonomy, across frameworks
- [Chunksets](/learn/chunking/chunksets) — what a root-to-leaf retrieval unit is and why it needs no overlap
- [The optimal chunker for LlamaParse](/optimal-chunker-llamaparse) — chunk a saved LlamaParse result without re-parsing
- [Document ingestion and chunking for RAG](/document-ingestion-chunking-rag) — the pipeline from file to answer