Source: http://www.poma-ai.com/docs/optimal-chunks-azure-ai-search

# Azure AI Search Chunking Strategy for RAG: The Optimal Chunks

<ByAuthor />

**The short answer:** Azure AI Search chunking is something you configure, not something you get. Inside an indexer pipeline the Text Split skill cuts text into fixed-length pages and the Document Layout skill cuts it into Markdown sections; both emit isolated fragments whose parent context survives only as a repeated string field. The optimal chunks for Azure AI Search are **chunksets** — a passage plus its full heading lineage — pushed straight to the index with `upload_documents`, with the hierarchy as filterable fields and the lineage text as both the vector source and the BM25 field, then retrieved with hybrid search and the semantic ranker.

## The Azure AI Search chunking that ships in the box

Microsoft's own framing is precise: *"In Azure AI Search, chunking is performed by skills and thus depends on indexers."* Nothing splits documents unless you build the pipeline. That pipeline, marketed as [integrated vectorization](https://learn.microsoft.com/en-us/azure/search/vector-search-integrated-vectorization), has four moving parts: a data source, a skillset containing a chunking skill plus an embedding skill, an index with a vector field, and an indexer to drive all of it.

Two skills do the cutting.

**Text Split skill** (`Microsoft.Skills.Text.SplitSkill`) is the default. `textSplitMode` is either `pages` or `sentences`. In `pages` mode, `maximumPageLength` sets the target size — minimum 300, maximum 50,000, default 5,000 — and the splitter backs off to a sentence boundary rather than truncating mid-sentence. `pageOverlapLength` copies that many characters from the end of the previous page into the next and must be under half the maximum. `maximumPagesToTake` caps how many chunks come off each document; `0`, the default, takes all of them. `unit` is `characters` by default, or `azureOpenAITokens`, in which case `azureOpenAITokenizerParameters.encoderModelName` picks the tokenizer (`cl100k_base` by default; `o200k_base` is not supported). Outputs are `textItems`, `offsets`, `lengths` and `ordinalPositions`. The skill is free and non-billable.

**Document Layout skill** (`Microsoft.Skills.Util.DocumentIntelligenceLayoutSkill`) calls the Azure Document Intelligence layout model. With `outputFormat` set to `markdown` it returns `markdown_document`: a collection of objects, each with `content`, an `ordinal_position`, and a `sections` dictionary whose keys run `h1` through the level set by `markdownHeaderDepth` (`h6` by default). With `outputFormat` set to `text` it returns `text_sections` and can chunk them itself via `chunkingProperties` (`unit: characters`, `maximumLength`, `overlapLength`), optionally with `locationMetadata` carrying `pageNumber` and bounding polygons.

Whatever a skill emits then has to become search documents. That is what [index projections](https://learn.microsoft.com/en-us/azure/search/search-how-to-define-index-projections) are for: `indexProjections.selectors` names a `targetIndexName`, a `parentKeyFieldName`, a `sourceContext` such as `/document/pages/*`, and an explicit `mappings` entry for every child field. Set `parameters.projectionMode` to `skipIndexingParentDocuments` unless you want parent rows with null chunk fields sitting in the same index.

## Where the built-in chunking stops

Both skills are competent at what they do. Neither decides what a retrieval unit should *mean*.

The Text Split skill is a character or token counter with a sentence-boundary heuristic. It has no view of the document at all: a `## Termination clauses` heading on page 41 and the paragraph that follows it land in different pages whenever the character count says so, and the paragraph keeps no record of the heading.

The Document Layout skill does better — it knows which `h1`/`h2`/`h3` a section sits under, and that is real structure. But the hierarchy arrives as a `sections` dictionary beside the content, and the moment you project it into an index you have to choose: drop it, or flatten it into a string field that repeats on every child row. Microsoft's recommended pattern is exactly that repetition — *"repeating parent fields in a single index"* — because the service has no query-time joins. The chunk's ancestry becomes decoration on the row rather than part of what gets embedded, and `markdownHeaderDepth` rolls everything deeper than your chosen level into one bucket.

So the vector you search over is built from an orphaned fragment. Overlap is the usual patch, and Microsoft's guidance reflects it: start at 512 tokens with 25 percent overlap, or `maximumPageLength` 2,000 with `pageOverlapLength` 500. Overlap duplicates spans into the HNSW graph and into the inverted index, inflates the document count (the docs' own table shows a NASA e-book going from 172 chunks to 216 when 200 characters of overlap are added at length 1,000), and still cannot tell a fragment which chapter it belongs to.

That matters more here than in most stores, because Azure AI Search's best retrieval mode depends on chunk quality. The [semantic ranker](https://learn.microsoft.com/en-us/azure/search/semantic-how-to-configure) reranks up to 50 merged results by reading the fields listed in the semantic configuration. Feed it fragments that never name their own subject and the L2 rerank has nothing to reason over.

## What "optimal chunks" means for Azure AI Search, concretely

1. **One search document per chunkset, self-explanatory alone.** A chunkset is a root-to-leaf path — `Master Services Agreement → Termination clauses → Early termination → "Either party may…"`. It reads correctly with no neighbours and no parent lookup. See [POMA chunksets](/learn/chunking/chunksets).
2. **Hierarchy as filterable fields, not as prose.** `file_id`, `chunkset_index` and `depth` marked `filterable` turn an OData `filter` into document-aware scoping. Filters run as `preFilter` by default, before the vector query, so they shrink the search surface instead of trimming results afterwards.
3. **The lineage text is the vector source.** Embed `to_embed`, the form that already carries the heading path, so the ancestry is inside the vector rather than in a sibling column.
4. **The same text in a searchable field.** Azure AI Search runs BM25 and the vector query in parallel and fuses them with Reciprocal Rank Fusion. Exact tokens — clause numbers, SKUs, defined terms — are caught lexically even when the embedding blurs them.
5. **No overlap.** Context is carried structurally, so there is nothing to stitch across a boundary.
6. **`k` = 50 when the semantic ranker is on.** The ranker accepts up to 50 inputs; giving it fewer starves it.

## The Azure AI Search RAG pipeline: PrimeCut chunksets pushed to the index

There is no POMA integration package for Azure AI Search, and none is needed — this is plain `azure-search-documents`. PrimeCut ingests the document (PDF, DOCX, HTML, images, or an existing parser's result JSON) and returns chunks and chunksets; the push API takes it from there.

```bash
pip install azure-search-documents poma
```

```python
import os

from azure.core.credentials import AzureKeyCredential
from azure.search.documents import SearchClient
from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
    HnswAlgorithmConfiguration, SearchField, SearchFieldDataType, SearchIndex,
    SemanticConfiguration, SemanticField, SemanticPrioritizedFields, SemanticSearch,
    VectorSearch, VectorSearchProfile,
)
from azure.search.documents.models import VectorizedQuery
from poma import PrimeCut

ENDPOINT = os.environ["AZURE_SEARCH_ENDPOINT"]
CREDENTIAL = AzureKeyCredential(os.environ["AZURE_SEARCH_ADMIN_KEY"])
DIMS = 1536  # must match your embedding model

# 1. Chunk any document — PDF, DOCX, HTML, or a bring-your-own-parser result JSON.
result = PrimeCut().ingest("contract.pdf")  # reads POMA_API_KEY
depth_of = {c.chunk_index: c.depth for c in result.chunks}

# 2. An index built around chunks: one vector field, the hierarchy as filterable fields.
index_client = SearchIndexClient(endpoint=ENDPOINT, credential=CREDENTIAL)
index_client.create_or_update_index(SearchIndex(
    name="contracts",
    fields=[
        SearchField(name="chunk_id", type=SearchFieldDataType.String,
                    key=True, filterable=True, analyzer_name="keyword"),
        SearchField(name="file_id", type=SearchFieldDataType.String,
                    filterable=True, facetable=True),
        SearchField(name="chunkset_index", type=SearchFieldDataType.Int32,
                    filterable=True, sortable=True),
        SearchField(name="depth", type=SearchFieldDataType.Int32, filterable=True),
        SearchField(name="chunk", type=SearchFieldDataType.String, searchable=True),
        SearchField(name="chunk_vector",
                    type=SearchFieldDataType.Collection(SearchFieldDataType.Single),
                    searchable=True, vector_search_dimensions=DIMS,
                    vector_search_profile_name="hnsw-profile"),
    ],
    vector_search=VectorSearch(
        algorithms=[HnswAlgorithmConfiguration(name="hnsw")],
        profiles=[VectorSearchProfile(name="hnsw-profile",
                                      algorithm_configuration_name="hnsw")],
    ),
    semantic_search=SemanticSearch(configurations=[
        SemanticConfiguration(
            name="chunkset-semantic",
            prioritized_fields=SemanticPrioritizedFields(
                content_fields=[SemanticField(field_name="chunk")],
            ),
        )
    ]),
))

# 3. One search document per chunkset. No skillset, no index projections.
search_client = SearchClient(endpoint=ENDPOINT, index_name="contracts",
                             credential=CREDENTIAL)
documents = [
    {
        "chunk_id": f"{cs.file_id}-{cs.chunkset_index}",
        "file_id": cs.file_id,
        "chunkset_index": cs.chunkset_index,
        "depth": depth_of[cs.chunks[-1]],  # leaf chunk
        "chunk": cs.to_embed,
        "chunk_vector": embed(cs.to_embed),  # your embedding model
    }
    for cs in result.chunksets
]
search_client.upload_documents(documents=documents)

# 4. Hybrid retrieval: BM25 + vector fused by RRF, reranked, scoped by filter.
query = "What are the early termination conditions?"
results = search_client.search(
    search_text=query,
    vector_queries=[VectorizedQuery(vector=embed(query),
                                    k_nearest_neighbors=50,
                                    fields="chunk_vector")],
    filter="file_id eq 'contract-001'",
    query_type="semantic",
    semantic_configuration_name="chunkset-semantic",
    select=["file_id", "chunkset_index", "chunk"],
    top=10,
)
for hit in results:
    print(hit["@search.score"], hit["chunk"][:120])
```

`upload_documents` is an upsert, so re-running the ingest for a changed source file overwrites the rows whose `chunk_id` is stable and leaves the rest alone. Deleting a source document means deleting its rows by `file_id` — the change tracking that an indexer gives you for free is yours to run.

Because retrieved chunksets from the same document share ancestor lineage, the hits can be merged along that shared path before they reach the prompt. On a legal document in our reference set, the same question was answered from 337 tokens of retrieved context with chunksets versus 1,542 with a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the [ingestion guide](/document-ingestion-chunking-rag) has the broader numbers.

## Three ways to chunk for Azure AI Search, compared

| | Text Split skill | Document Layout skill | **Chunksets pushed to the index** |
| --- | --- | --- | --- |
| Where it runs | Skillset, indexer-driven | Skillset, indexer-driven | Your code, before indexing |
| Cut boundaries | Character or token count, sentence-aware | Markdown headings to `markdownHeaderDepth` | Document hierarchy, root to leaf |
| Chunk is self-explanatory alone | ✗ | Only within its section | **✓ by construction** |
| Ancestry available at query time | ✗ | As a repeated `sections`/parent field | **✓ inside the embedded text and as `depth`** |
| Overlap needed | Yes (`pageOverlapLength`) | Yes in `text` mode (`overlapLength`) | **None** |
| Tables | Cut wherever the count lands | Preserved in Markdown | **Kept whole** |
| Wiring required | Data source, skillset, `indexProjections`, indexer | Same, plus a billable Foundry resource | **`upload_documents`** |
| Embedding model | Skill-bound (Azure OpenAI, AML, custom skill) | Same | **Any model you can call** |
| Billing | Free, non-billable | Foundry Tools Standard beyond 20 docs/indexer/day | **Your embedding calls only** |

The two routes are not exclusive. The indexer pipeline earns its keep when your corpus lives in Blob Storage or OneLake and changes constantly, because change tracking and deletion detection come with it. The push route earns its keep when chunk quality decides retrieval quality — which, for RAG over structured documents, it usually does.

## Feeding it from the parser you already run

PrimeCut does ingestion and chunking, and it is a direct alternative to text splitters. If you already run a parser or OCR engine, it will take that engine's output instead and do chunking only: result JSON from Azure Document Intelligence, AWS Textract, LlamaParse, Mistral OCR, Unstructured, Docling, Marker or PaddleOCR-VL is auto-detected by shape. Markdown from any other tool goes in as a `.md` file. Either way the output is the same `result.chunksets`, and step 3 above is unchanged.

Note the overlap with Azure's own stack: the Document Layout skill and Azure Document Intelligence are the same layout model. If you already call Document Intelligence directly, keep it — see [the Azure Document Intelligence chunking guide](/optimal-chunker-azure-document-intelligence) for feeding its `analyzeResult` into chunksets.

## Frequently asked questions

### Does Azure AI Search do chunking automatically?

Only inside an indexer pipeline, and only if you configure it. Chunking in Azure AI Search is performed by skills, so it depends on an indexer, a data source, and a skillset. Add the Text Split skill and the service splits text into pages or sentences; add the Document Layout skill and it returns Markdown sections. If you push documents to the index yourself with the upload API, nothing is split for you: whatever you upload is one search document.

### What is the best Azure AI Search chunking strategy for RAG?

Index one search document per self-explanatory unit, with the document hierarchy stored as filterable fields rather than implied by the text. In practice that means chunksets: a passage plus its full heading lineage, embedded into a Collection(Edm.Single) field with a vectorSearchProfile, alongside file_id, chunkset_index and depth as filterable fields and the chunk text in a searchable field for BM25. Then query with hybrid search and the semantic ranker.

### What are the Text Split skill parameters and what should I set them to?

The @odata.type is Microsoft.Skills.Text.SplitSkill. textSplitMode is pages or sentences. For pages, maximumPageLength sets the chunk size (minimum 300, maximum 50000, default 5000 characters), pageOverlapLength copies that many characters or tokens from the end of the previous page and must be less than half the maximum, and maximumPagesToTake limits how many chunks are taken per document (0 means all). unit is characters by default or azureOpenAITokens, with azureOpenAITokenizerParameters.encoderModelName selecting the tokenizer (cl100k_base by default). Microsoft recommends starting at maximumPageLength 2000 with pageOverlapLength 500.

### Azure AI Search skillset chunking: how do I customize it?

Two routes. Inside the indexer pipeline, call a custom Web API skill that receives the document text and returns your own chunks, then project them with indexProjections. Outside the pipeline, chunk before indexing and push one search document per chunk with SearchClient.upload_documents from the azure-search-documents package. The push route is simpler, has no skillset to debug, and lets you compute the embedding with any model you like.

### Azure AI Search semantic chunking: is it supported?

Not as a Text Split skill mode. The closest built-in options are the Document Layout skill, which returns Markdown sections keyed by heading up to markdownHeaderDepth, and the Azure Content Understanding skill, which Microsoft documents as producing semantic chunks with Markdown output. Both are structure-aware rather than embedding-similarity based. Anything beyond that is a custom skill or pre-computed chunks pushed to the index.

### Do I still need index projections if I push my own chunks?

No. Index projections exist to solve one-to-many indexing inside an indexer run, mapping one parent document to many child documents through selectors, parentKeyFieldName and sourceContext. If you chunk before indexing, you already have the many side and you write it directly with upload_documents. Keep a filterable parent field such as file_id so you can still scope queries to one source document, and handle re-indexing of changed documents yourself.

### What chunk size and overlap should I use in Azure AI Search?

Microsoft recommends starting at 512 tokens with 25 percent overlap, or 2000 characters with 500 characters of overlap for the Text Split skill. Those are sensible defaults for fixed-size splitting, but the size knob cannot add context that the chunk never had. A chunk that carries its own heading lineage is self-explanatory at any size and needs no overlap, which also keeps the HNSW graph and the inverted index free of duplicated spans.

### How does the semantic ranker interact with chunk design?

The semantic ranker reranks up to 50 documents returned by the hybrid query, reading the fields named in the semantic configuration. It reads chunk text, not the surrounding document, so a fragment that does not name its own subject gives the reranker nothing to work with. Set k to 50 on the vector query so the ranker gets a full input set, and put the field that carries the heading lineage in content_fields.

Running a different store? The same chunk design applies: [Chroma](/optimal-chunks-chroma) · [Elasticsearch](/optimal-chunks-elasticsearch) · [FAISS](/optimal-chunks-faiss) · [LanceDB](/optimal-chunks-lancedb) · [Milvus](/optimal-chunks-milvus) · [MongoDB Atlas](/optimal-chunks-mongodb-atlas) · [OpenSearch](/optimal-chunks-opensearch) · [pgvector](/optimal-chunks-pgvector) · [Pinecone](/optimal-chunks-pinecone) · [Qdrant](/optimal-chunks-qdrant) · [Redis](/optimal-chunks-redis) · [Turbopuffer](/optimal-chunks-turbopuffer) · [Vespa](/optimal-chunks-vespa) · [Weaviate](/optimal-chunks-weaviate) — or browse [all pipeline recipes](/pipelines/).

## Continue reading

- [POMA chunksets](/learn/chunking/chunksets) — what a root-to-leaf retrieval unit is and why it needs no overlap
- [RAG chunking strategies and text splitters](/rag-chunking-strategies-text-splitters) — the full taxonomy, fixed-size to hierarchy-aware
- [Document ingestion and chunking for RAG](/document-ingestion-chunking-rag) — the illustration behind the 337 vs 1,542 token figure
- [Azure Document Intelligence chunking](/optimal-chunker-azure-document-intelligence) — feeding layout output into chunksets
- [RAG architecture guide](/guides/rag-architecture/) — where chunking sits in the wider pipeline