Source: http://www.poma-ai.com/docs/optimal-chunks-qdrant

# The Optimal Chunks for the Best Retrieval in Qdrant

<ByAuthor />

**The short answer:** the optimal chunks for Qdrant are **chunksets** — self-explanatory units that carry their full heading lineage — embedded as **hybrid dense + sparse points** with hierarchy metadata (`file_id`, `page`, `depth`) in an indexed payload. Qdrant's search is as good as vector search gets; what decides your retrieval quality is what you put *into* it. POMA's first-party `PomaQdrant` integration writes exactly this shape in one call and assembles retrieved points back into prompt-ready cheatsheets.

This page covers what Qdrant does brilliantly, what it cannot do for you, and the point/payload design that gets the most out of it.

## Qdrant gives you world-class search — over whatever you feed it

Qdrant's primitives are a gift to RAG engineers:

- **Named vectors** — several embeddings per point (dense + sparse, or multiple models).
- **Sparse vectors** — BM25-style term matching lives *inside* the same index, so hybrid retrieval is one query, not two systems.
- **Payload indexes** — filter by document, tenant, page, or any field at ANN speed.
- **HNSW with quantization options** — scalar, product, and binary quantization when the collection grows.

None of these primitives, however, know anything about your documents. Qdrant retrieves the nearest neighbors of what you embedded. If what you embedded is a context-free fragment — a paragraph cut loose from the section that gives it meaning — Qdrant will faithfully retrieve context-free fragments, at any scale, with excellent latency. **The ceiling on retrieval quality is set before the first vector is upserted.**

## What "optimal chunks" means for Qdrant, concretely

Four properties separate a Qdrant collection that answers questions from one that returns trivia:

1. **Every point is self-explanatory.** A chunkset is a root-to-leaf path through the document — `Master Services Agreement → Termination clauses → Early termination → "Either party may…"`. Retrieved alone, it still tells the LLM (and the human reading citations) exactly where it sits. A bare 512-token window does not. See [POMA chunksets](/learn/chunking/chunksets).
2. **No overlap, no near-duplicates.** Overlap is a patch for splitters that cut mid-thought — and it embeds every overlapped span twice. That inflates the HNSW graph and, worse, fills top-k with near-identical hits. Chunksets carry context structurally, so overlap is simply unnecessary and your `limit=5` returns five *different* candidates.
3. **Hierarchy in the payload, indexed.** `file_id`, `page`, `depth`, `chunk_index` as payload fields turn Qdrant filters into document-aware operations: scope a query to one contract, exclude appendices by depth, cite pages in the answer.
4. **Hybrid by default.** Legal, financial, and technical corpora are full of exact tokens (§ numbers, SKUs, defined terms) that dense embeddings smear. A sparse BM25 vector on the same point catches them; Qdrant fuses the two result sets server-side.

## The shortest path: PomaQdrant

POMA ships a first-party Qdrant integration — `PomaQdrant` is a `QdrantClient` subclass, so nothing about Qdrant is hidden from you:

```bash
pip install 'poma[qdrant]'
```

```python
import os

from poma import PrimeCut
from poma.integrations.qdrant import PomaQdrant

# 1. Chunk any document — PDF, DOCX, HTML, or a bring-your-own-OCR result JSON.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.pdf")

# 2. Write chunksets as hybrid dense+sparse points with hierarchy payloads.
qdrant = PomaQdrant(
    url=os.environ["QDRANT_URL"],
    api_key=os.environ["QDRANT_API_KEY"],
    cloud_inference=True,
    collection_name="contracts",
    dense_model="sentence-transformers/all-MiniLM-L6-v2",
    sparse_model="Qdrant/bm25",   # hybrid on by default
    dense_size=384,
    auto_create_collection=True,
)
qdrant.upsert_poma_points(result)

# 3. Retrieve + assemble prompt-ready context in one call.
cheatsheets = qdrant.get_cheatsheets(
    query="What does the contract say about early termination?",
    limit=3,
)
print(cheatsheets[0]["content"])
```

`upsert_poma_points(...)` accepts a typed `PomaResult`, legacy chunk-data dictionaries, or a path to a `.poma` archive. `get_cheatsheets(...)` also takes a raw `query_obj`/`prefetch` for full Qdrant query control, or `results=...` to convert searches you ran yourself. Full signatures: [Qdrant integration reference](/sdk/reference/qdrant).

The cheatsheet step is where the token savings land: retrieved chunksets share ancestors, and merging them deduplicates the lineage into one coherent block. On our reference legal-document benchmark that meant **337 tokens** of context versus **1,542** for a recursive-splitter baseline, with zero information loss — [methodology here](/document-ingestion-chunking-rag).

## Chunk shapes in Qdrant, compared

| Property | Fixed 512-token chunks + overlap | Semantic chunks | **POMA chunksets** |
| --- | --- | --- | --- |
| Point is self-explanatory when retrieved alone | ✗ | Sometimes | **✓ (by construction)** |
| Near-duplicate vectors in the index | Many (overlap) | Few | **None** |
| Page + hierarchy in payload | Manual | Manual | **✓ (`file_id`, `page`, `depth`)** |
| Tables survive embedding | ✗ (cut mid-row) | Fragile | **✓ (HTML, never cut)** |
| Hybrid dense+sparse points | DIY | DIY | **✓ default in `PomaQdrant`** |
| Post-retrieval assembly | Concatenate raw hits | Concatenate raw hits | **Cheatsheets (deduped lineage)** |

## Frequently asked questions

### What is the optimal chunk size for Qdrant?

There is no universal token count — the optimal unit is a chunk that is self-explanatory without its neighbors. Chunksets achieve that at any size, which is why they beat fixed 512-token windows: every point can be retrieved alone and still be understood. If you must fix a size, start at 512 tokens — but the size knob cannot fix missing context.

### What should I store in Qdrant payloads for RAG?

`file_id`, `page`, `depth`, `chunk_index`, plus the chunk text. Payload-index the fields you filter by (`file_id`, `page` at minimum). That enables per-document scoping, page citations, and depth-aware re-ranking.

### Should I use hybrid search in Qdrant for RAG?

For most document corpora, yes. Sparse BM25 vectors catch exact terms dense embeddings blur; Qdrant hosts both on the same point and fuses results. `PomaQdrant` writes both by default.

### How does chunk overlap affect a Qdrant index?

It embeds every overlapped span twice: larger index, slower HNSW build, and near-duplicate hits crowding out diverse results. Chunks that carry hierarchy don't need overlap.

### How do I get from Qdrant results to prompt-ready context?

Merge, don't concatenate: deduplicate the shared ancestor lineage across retrieved chunksets and assemble one coherent block. `get_cheatsheets(query=...)` does search + assembly in one call.

### Does POMA have a native Qdrant integration?

Yes — `pip install 'poma[qdrant]'`, then `PomaQdrant.upsert_poma_points(result)` and `get_cheatsheets(query=...)`. It subclasses `QdrantClient`, so all of Qdrant stays available.

## Retrieval with vektoria's assemble()

`PomaQdrant` above is the full-fat, Qdrant-specific integration. POMA also ships a thinner, DB-agnostic layer — `poma.vektoria` — built around one rule: the vector DB stores only `(id, vector, {file_id, chunkset_index})`; the actual chunk content lives on a **volume** (a path, `s3://`, or `gs://` URL) via `poma.vektoria.Volume`. Retrieval is your normal Qdrant query, handed to `assemble()`:

```python
from poma.vektoria import assemble

resp = client.query_points("poma", query=qv, limit=10, with_payload=True)
context = assemble(resp, volume="s3://your-bucket/poma")  # -> [{"file_id", "content"}, ...]
```

`assemble()` auto-detects the result shape (Qdrant among 12 supported stores) and returns deduplicated cheatsheets — the same content-free-index discipline, without `PomaQdrant`'s Qdrant-specific upsert helpers. Use whichever fits: `PomaQdrant` for a batteries-included Qdrant integration, `vektoria` when Qdrant is one of several stores you retrieve from.

## Feed Qdrant from the parser you already run

Pair this page with your ingestion side — the same chunksets, straight from your OCR/parsing tool's raw output:

- [Mistral OCR → Qdrant](/pipelines/mistral-ocr-to-qdrant)
- [LlamaParse → Qdrant](/pipelines/llamaparse-to-qdrant)
- [Azure Document Intelligence → Qdrant](/pipelines/azure-document-intelligence-to-qdrant)
- [Unstructured.io → Qdrant](/pipelines/unstructured-to-qdrant)
- [Docling → Qdrant](/pipelines/docling-to-qdrant)
- [Marker → Qdrant](/pipelines/marker-to-qdrant)
- [AWS Textract → Qdrant](/pipelines/textract-to-qdrant)

Running a different store? The same chunk design applies: [Pinecone](/optimal-chunks-pinecone) · [Weaviate](/optimal-chunks-weaviate) · [pgvector](/optimal-chunks-pgvector) · [Milvus](/optimal-chunks-milvus) · [Chroma](/optimal-chunks-chroma).

Fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [RAG architecture guide](/guides/rag-architecture/) · [POMA chunksets](/learn/chunking/chunksets).