Source: http://www.poma-ai.com/docs/optimal-chunks-milvus

# The Optimal Chunks for the Best Retrieval in Milvus

<ByAuthor />

**The short answer:** the optimal chunks for Milvus are **chunksets** — self-explanatory units that carry their full heading lineage — embedded from their `to_embed` text, stored with hierarchy metadata (`file_id`, `page`, `depth`) as **scalar fields** for filtered search, and queried **hybrid dense + sparse** where your Milvus version supports it. Milvus scales vector search further than almost anything else; what decides your retrieval quality is what you put *into* the collection. POMA's PrimeCut produces exactly this shape, and retrieved chunksets merge client-side into prompt-ready cheatsheets.

This page covers what Milvus does brilliantly, what it cannot do for you, and the collection design that gets the most out of it.

## Milvus gives you search at serious scale — over whatever you feed it

Milvus's primitives are built for collections that keep growing:

- **Collections and partitions** — a partition key hashes entities into partitions automatically, so multi-tenant and per-corpus isolation come without a collection per tenant.
- **Scalar fields with filtered search** — boolean expressions (`file_id == "..."`, `depth <= 2`) are applied *during* the ANN search, not as a post-filter afterthought.
- **A spectrum of index types** — HNSW for memory-resident speed, IVF variants for smaller footprints, DiskANN for corpora that outgrow RAM.
- **Sparse vectors and BM25 hybrid** — recent Milvus versions host sparse term vectors next to dense embeddings and fuse both in one hybrid request.
- **Zilliz Cloud** — the same API, managed.

None of these primitives, however, know anything about your documents. Milvus retrieves the nearest neighbors of what you embedded. If what you embedded is a context-free fragment — a paragraph cut loose from the section that gives it meaning — Milvus will faithfully retrieve context-free fragments, at any scale, with excellent latency. **The ceiling on retrieval quality is set before the first entity is inserted.**

## What "optimal chunks" means for Milvus, concretely

Four properties separate a Milvus collection that answers questions from one that returns trivia:

1. **Every entity is self-explanatory.** A chunkset is a root-to-leaf path through the document — `Master Services Agreement → Termination clauses → Early termination → "Either party may…"`. Retrieved alone, it still tells the LLM (and the human reading citations) exactly where it sits. A bare 512-token window does not. See [POMA chunksets](/learn/chunking/chunksets).
2. **No overlap, no near-duplicates.** Overlap is a patch for splitters that cut mid-thought — and it embeds every overlapped span twice. That inflates the entity count (and with it HNSW build time, IVF training, or DiskANN's disk footprint) and fills top-k with near-identical hits. Chunksets carry context structurally, so overlap is unnecessary and your `limit=5` returns five *different* candidates.
3. **Hierarchy in scalar fields.** `file_id`, `page`, `depth`, `chunk_index` as scalar fields turn Milvus filter expressions into document-aware operations: scope a query to one contract, exclude appendices by depth, cite pages in the answer. Promote `file_id` (or `tenant_id`) to the partition key and the same field also drives partition routing.
4. **Hybrid where it counts.** Legal, financial, and technical corpora are full of exact tokens (§ numbers, SKUs, defined terms) that dense embeddings smear. On recent Milvus versions, a sparse BM25 vector in the same collection catches them, and one hybrid request fuses both result sets.

## Chunksets into Milvus with pymilvus

There is no shortcut integration to hide here — the point is that POMA's output maps one-to-one onto a clean Milvus schema:

```python
import os

from poma import PrimeCut
from pymilvus import DataType, MilvusClient

# 1. Chunk any document — PDF, DOCX, HTML, or a bring-your-own-OCR result JSON.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.pdf")

# 2. A chunkset collection: one dense vector + hierarchy scalar fields.
milvus = MilvusClient(
    uri=os.environ["MILVUS_URI"],
    token=os.environ["MILVUS_TOKEN"],
)

schema = MilvusClient.create_schema(auto_id=True)
schema.add_field("pk", DataType.INT64, is_primary=True)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=384)
schema.add_field("text", DataType.VARCHAR, max_length=65535)
schema.add_field("file_id", DataType.VARCHAR, max_length=128)
schema.add_field("page", DataType.INT64)
schema.add_field("depth", DataType.INT64)
schema.add_field("chunk_index", DataType.INT64)

index_params = milvus.prepare_index_params()
index_params.add_index(field_name="vector", index_type="HNSW", metric_type="COSINE")

milvus.create_collection(
    collection_name="chunksets",
    schema=schema,
    index_params=index_params,
)

# 3. Embed each chunkset's normalized to_embed text and insert.
milvus.insert(
    collection_name="chunksets",
    data=[
        {
            "vector": embed(cs.to_embed),  # your embedding model
            "text": cs.to_embed,
            "file_id": cs.file_id,
            "page": cs.page,
            "depth": cs.depth,
            "chunk_index": cs.chunk_index,
        }
        for cs in result.chunksets
    ],
)

# 4. Filtered search: scope the ANN query to one document.
hits = milvus.search(
    collection_name="chunksets",
    data=[embed("What does the contract say about early termination?")],
    filter='file_id == "contract.pdf"',
    limit=5,
    output_fields=["text", "file_id", "page", "depth", "chunk_index"],
)
```

On recent Milvus versions, add a `SPARSE_FLOAT_VECTOR` field with a BM25 function on the same schema and issue a hybrid request that fuses dense and sparse rankings — the chunkset design does not change, only the query does. For multi-tenant deployments, declare `file_id` (or a `tenant_id` field) with `is_partition_key=True` and Milvus routes filtered queries to the right partitions automatically.

The last step is client-side: retrieved chunksets share ancestors, so deduplicate the shared lineage and merge the hits into one coherent context block — a **cheatsheet** — before it goes into the prompt. On our reference legal-document benchmark that assembly meant **337 tokens** of context versus **1,542** for a recursive-splitter baseline, with zero information loss — [methodology here](/document-ingestion-chunking-rag).

## Chunk shapes in Milvus, compared

| Property | Fixed 512-token chunks + overlap | Semantic chunks | **POMA chunksets** |
| --- | --- | --- | --- |
| Entity is self-explanatory when retrieved alone | ✗ | Sometimes | **✓ (by construction)** |
| Near-duplicate vectors in the index | Many (overlap) | Few | **None** |
| Hierarchy in scalar fields for filtered search | Manual | Manual | **✓ (`file_id`, `page`, `depth`)** |
| Partition-key multi-tenancy fits naturally | Retrofit | Retrofit | **✓ (`file_id`/`tenant_id` scalar)** |
| Tables survive embedding | ✗ (cut mid-row) | Fragile | **✓ (HTML, never cut)** |
| Fits DiskANN-scale corpora without bloat | Index inflated by overlap | Moderate | **✓ (overlap-free, fewer entities)** |
| Post-retrieval assembly | Concatenate raw hits | Concatenate raw hits | **Cheatsheets (deduped lineage)** |

## Frequently asked questions

### What is the optimal chunk size for Milvus?

There is no universal token count — the optimal unit is an entity that is self-explanatory without its neighbors. Chunksets achieve that at any size, which is why they beat fixed 512-token windows: every entity can be retrieved alone and still be understood. If you must fix a size, start at 512 tokens — but the size knob cannot fix missing context.

### What scalar fields should I add to a Milvus collection for RAG?

`file_id`, `page`, `depth`, `chunk_index`, plus a VARCHAR field for the chunk text. Milvus applies filter expressions like `file_id == "contract-001"` or `depth <= 2` during the ANN search itself, enabling per-document scoping, page citations, and depth-aware re-ranking without a second lookup.

### Should I use hybrid search in Milvus for RAG?

For most document corpora, yes. Recent Milvus versions host sparse BM25-style vectors next to dense embeddings in the same collection and fuse both in one hybrid request. Sparse catches exact terms dense embeddings blur; dense catches paraphrases. One `to_embed` text feeds both.

### How do I make a Milvus collection multi-tenant?

Declare a scalar field (`tenant_id`, or `file_id` for per-document isolation) as the partition key at collection creation. Milvus hashes entities into partitions automatically and routes filtered queries to only the relevant ones — isolation and lower latency without a collection per tenant.

### Which Milvus index type should I use for RAG — HNSW, IVF, or DiskANN?

HNSW for memory-resident speed and recall; IVF variants for smaller footprints; DiskANN when the corpus outgrows RAM and NVMe-resident indexing is worth a modest latency cost. Overlap-free chunksets keep the entity count down, which makes every one of the three cheaper.

### How do I turn Milvus search results into prompt-ready context?

Merge, don't concatenate: deduplicate the shared ancestor lineage across retrieved chunksets and assemble one coherent block — a cheatsheet — client-side. That assembly is how the reference benchmark answers with 337 tokens instead of 1,542.

## Retrieval with vektoria's assemble()

POMA's `poma.vektoria` package keeps the collection content-free: Milvus stores only `(id, vector, {file_id, chunkset_index})`, while the actual chunk content lives on a **volume** (a path, `s3://`, or `gs://` URL). After your normal Milvus search, hand the raw result to `assemble()`:

```python
from poma.vektoria import assemble

res = client.search("poma", data=[qv], limit=10, output_fields=["file_id", "chunkset_index"])
context = assemble(res, volume="s3://your-bucket/poma")  # -> [{"file_id", "content"}, ...]
```

`assemble()` auto-detects Milvus's result shape (one of 12 supported stores) and returns deduplicated cheatsheets. The one requirement: `output_fields=["file_id", "chunkset_index"]` on the search, or Milvus won't return them for `assemble()` to work with.

## Feed Milvus from the parser you already run

Pair this page with your ingestion side — the same chunksets, straight from your OCR/parsing tool's raw output:

- [Mistral OCR → Milvus](/pipelines/mistral-ocr-to-milvus)
- [LlamaParse → Milvus](/pipelines/llamaparse-to-milvus)
- [Azure Document Intelligence → Milvus](/pipelines/azure-document-intelligence-to-milvus)
- [Unstructured.io → Milvus](/pipelines/unstructured-to-milvus)
- [Docling → Milvus](/pipelines/docling-to-milvus)
- [Marker → Milvus](/pipelines/marker-to-milvus)
- [AWS Textract → Milvus](/pipelines/textract-to-milvus)

Running a different store? The same chunk design applies: [Qdrant](/optimal-chunks-qdrant) · [Pinecone](/optimal-chunks-pinecone) · [Weaviate](/optimal-chunks-weaviate) · [pgvector](/optimal-chunks-pgvector) · [Chroma](/optimal-chunks-chroma).

Fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [RAG architecture guide](/guides/rag-architecture/) · [POMA chunksets](/learn/chunking/chunksets).