Skip to content

Qdrant Chunking Strategy for RAG: Late, Semantic, and Chunksets ​

The short answer: Qdrant chunking is not a token-count problem. The optimal chunks for Qdrant are chunksets — self-explanatory units that carry their full heading lineage — embedded as hybrid dense + sparse points with hierarchy metadata (file_id, page, depth) in an indexed payload. Qdrant's search is as good as vector search gets; what decides your retrieval quality is what you put into it. POMA's first-party PomaQdrant integration writes exactly this shape in one call and assembles retrieved points back into prompt-ready cheatsheets.

This page covers what Qdrant does brilliantly, what it cannot do for you, and the point/payload design that gets the most out of it.

Qdrant gives you world-class search — over whatever you feed it ​

Qdrant's primitives are a gift to RAG engineers:

  • Named vectors — several embeddings per point (dense + sparse, or multiple models).
  • Sparse vectors — BM25-style term matching lives inside the same index, so hybrid retrieval is one query, not two systems.
  • Payload indexes — filter by document, tenant, page, or any field at ANN speed.
  • HNSW with quantization options — scalar, product, and binary quantization when the collection grows.

None of these primitives, however, know anything about your documents. Qdrant retrieves the nearest neighbors of what you embedded. If what you embedded is a context-free fragment — a paragraph cut loose from the section that gives it meaning — Qdrant will faithfully retrieve context-free fragments, at any scale, with excellent latency. The ceiling on retrieval quality is set before the first vector is upserted.

What "optimal chunks" means for Qdrant, concretely ​

Four properties separate a Qdrant collection that answers questions from one that returns trivia:

  1. Every point is self-explanatory. A chunkset is a root-to-leaf path through the document — Master Services Agreement → Termination clauses → Early termination → "Either party may…". Retrieved alone, it still tells the LLM (and the human reading citations) exactly where it sits. A bare 512-token window does not. See POMA chunksets.
  2. No overlap, no near-duplicates. Overlap is a patch for splitters that cut mid-thought — and it embeds every overlapped span twice. That inflates the HNSW graph and, worse, fills top-k with near-identical hits. Chunksets carry context structurally, so overlap is simply unnecessary and your limit=5 returns five different candidates.
  3. Hierarchy in the payload, indexed. file_id, chunkset_index, page, depth as payload fields turn Qdrant filters into document-aware operations: scope a query to one contract, exclude appendices by depth, cite pages in the answer.
  4. Hybrid by default. Legal, financial, and technical corpora are full of exact tokens (§ numbers, SKUs, defined terms) that dense embeddings smear. A sparse BM25 vector on the same point catches them; Qdrant fuses the two result sets server-side.

The shortest path: PomaQdrant ​

POMA ships a first-party Qdrant integration — PomaQdrant is a QdrantClient subclass, so nothing about Qdrant is hidden from you:

bash
pip install 'poma[qdrant]'
python
import os

from poma import PrimeCut
from poma.integrations.qdrant import PomaQdrant

# 1. Chunk any document — PDF, DOCX, HTML, or a bring-your-own-OCR result JSON.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.pdf")

# 2. Write chunksets as hybrid dense+sparse points with hierarchy payloads.
qdrant = PomaQdrant(
    url=os.environ["QDRANT_URL"],
    api_key=os.environ["QDRANT_API_KEY"],
    cloud_inference=False,  # True needs Qdrant Cloud's inference service; False embeds locally via fastembed and works on OSS/local Qdrant too
    collection_name="contracts",
    dense_model="sentence-transformers/all-MiniLM-L6-v2",
    sparse_model="Qdrant/bm25",   # hybrid on by default
    dense_size=384,
    auto_create_collection=True,
)
qdrant.upsert_poma_points(result)

# 3. Retrieve + assemble prompt-ready context in one call.
cheatsheets = qdrant.get_cheatsheets(
    query="What does the contract say about early termination?",
    limit=3,
)
print(cheatsheets[0]["content"])

upsert_poma_points(...) accepts a typed PomaResult, legacy chunk-data dictionaries, or a path to a .poma archive. get_cheatsheets(...) also takes a raw query_obj/prefetch for full Qdrant query control, or results=... to convert searches you ran yourself. Full signatures: Qdrant integration reference.

The cheatsheet step is where the token savings land: retrieved chunksets share ancestors, and merging them deduplicates the lineage into one coherent block. On one reference legal document that meant 337 tokens of context versus 1,542 for a recursive-splitter baseline, with nothing lost. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.

Chunk shapes in Qdrant, compared ​

PropertyFixed 512-token chunks + overlapSemantic chunksPOMA chunksets
Point is self-explanatory when retrieved alone✗Sometimes✓ (by construction)
Near-duplicate vectors in the indexMany (overlap)FewNone
Page + hierarchy in payloadManualManual✓ (file_id, page, depth)
Tables survive embedding✗ (cut mid-row)Fragile✓ (HTML, never cut)
Hybrid dense+sparse pointsDIYDIY✓ default in PomaQdrant
Post-retrieval assemblyConcatenate raw hitsConcatenate raw hitsCheatsheets (deduped lineage)

Qdrant semantic chunking: a better cut, still a context-free point ​

Semantic chunking embeds each sentence, compares consecutive embeddings, and splits where similarity drops, on the theory that a trough marks a topic boundary. Qdrant's own course walks the same ladder — fixed-size, sentence, paragraph, sliding window, recursive, semantic — and is honest about the price: you embed the whole document up front purely to decide where to cut, and the similarity threshold is a per-corpus hyperparameter.

It is a genuine improvement over a 512-token window. It is also only half the problem. A semantically clean paragraph about early termination is a paragraph that never says which agreement it belongs to. Stored as a point and returned alone in a limit=5 result, it is an orphan: the payload can still report file_id and page, but the text the model reads starts mid-argument.

The unit is the lever semantic chunking never pulls. A chunkset is a root-to-leaf path, so the point is self-explanatory before the payload adds anything. Payload then does what payload is good at — create_payload_index on file_id with field_schema="keyword" and on page with "integer", so filters run at ANN speed, and set_payload when you need to backfill fields onto points that already exist.

Qdrant late chunking: a better vector for the same span ​

Late chunking is worth naming precisely, because it is routinely confused with semantic chunking and is not the same operation. It comes from Jina AI (arXiv:2409.04701), and it inverts the usual order:

  1. Feed the whole document through a long-context embedding model in one pass. jina-embeddings-v2-base-en and its small sibling take 8,192 tokens, about ten pages.
  2. Keep the token-level output — one vector per token, each already conditioned on the entire text.
  3. Apply your chunk boundaries after the transformer, and mean-pool the token vectors inside each boundary.

You end up with one vector per chunk, exactly as before, except each vector was computed with the rest of the document in view. A pronoun referring to a company named three paragraphs earlier embeds as though it knew that. Pooling each chunk in isolation cannot do this.

Two consequences for Qdrant. First, late chunking happens in your embedding step, not in the database. Qdrant stores whatever vector you hand it, so a late-chunked vector upserts through the same PointStruct as any other, and named vectors let it sit beside a sparse BM25 vector on the same point. There is nothing to switch on server-side. Second, the technique is bounded by the embedder's context window: past 8,192 tokens you are splitting the document first anyway, and the pooling only carries context within a single pass.

Late chunking and chunksets are orthogonal, and they compose. Late chunking improves the vector for a given span of text. Chunksets change the span — what the retrieval unit is and what it carries with it — so the text the LLM finally reads is self-explanatory and the payload has a hierarchy to filter on. You can run both: chunkset.to_embed is just a string, and chunkset boundaries are just spans over the document, so they can serve as the pooling boundaries in a late-chunking pass instead of fixed windows. What late chunking cannot do is fix the retrieved text, because it never touches it. The vector gets smarter; the chunk that lands in your prompt stays exactly as context-free as you cut it.

Frequently asked questions ​

What is the optimal chunk size for Qdrant? ​

There is no universal token count — the optimal unit is a chunk that is self-explanatory without its neighbors. Chunksets achieve that at any size, which is why they beat fixed 512-token windows: every point can be retrieved alone and still be understood. If you must fix a size, start at 512 tokens — but the size knob cannot fix missing context.

What should I store in Qdrant payloads for RAG? ​

file_id, chunkset_index, page, depth, plus the chunk text. page comes from the chunk records in the .poma archive (the chunkset’s leaf chunk); the SDK objects expose depth but not page. Payload-index the fields you filter by (file_id, page at minimum). That enables per-document scoping, page citations, and depth-aware re-ranking.

Should I use hybrid search in Qdrant for RAG? ​

For most document corpora, yes. Sparse BM25 vectors catch exact terms dense embeddings blur; Qdrant hosts both on the same point and fuses results. PomaQdrant writes both by default.

How does chunk overlap affect a Qdrant index? ​

It embeds every overlapped span twice: larger index, slower HNSW build, and near-duplicate hits crowding out diverse results. Chunks that carry hierarchy don't need overlap.

How do I get from Qdrant results to prompt-ready context? ​

Merge, don't concatenate: deduplicate the shared ancestor lineage across retrieved chunksets and assemble one coherent block. get_cheatsheets(query=...) does search + assembly in one call.

Does POMA have a native Qdrant integration? ​

Yes — pip install 'poma[qdrant]', then PomaQdrant.upsert_poma_points(result) and get_cheatsheets(query=...). It subclasses QdrantClient, so all of Qdrant stays available.

What is late chunking, and how do I use it with Qdrant? ​

Late chunking (Jina AI) embeds the whole document with a long-context model, keeps the token-level vectors, then applies chunk boundaries and mean-pools within each one — so every chunk vector is conditioned on the full text. It runs in your embedding step, not inside Qdrant: the result upserts through the ordinary point API and can sit beside a sparse vector as a named vector. Its ceiling is the embedder's context window (8,192 tokens for jina-embeddings-v2). It improves the vector without changing the retrieved text, so it composes with chunksets rather than competing with them.

Should I use semantic chunking as my Qdrant chunking strategy? ​

It beats a fixed 512-token window, at the cost of a full embedding pass up front and a threshold to tune per corpus. But it only picks better cut points: the resulting point still never states which document or section it came from. Chunksets change the unit rather than the cut, so each point carries its root-to-leaf heading path. The two are compatible.

Retrieval with vektoria's assemble() ​

PomaQdrant above is the full-fat, Qdrant-specific integration. POMA also ships a thinner, DB-agnostic layer — poma.vektoria — built around one rule: the vector DB stores only (id, vector, {file_id, chunkset_index}); the actual chunk content lives on a volume (a path, s3://, or gs:// URL) via poma.vektoria.Volume. Retrieval is your normal Qdrant query, handed to assemble():

python
from poma.vektoria import assemble

resp = client.query_points("poma", query=qv, limit=10, with_payload=True)
context = assemble(resp, volume="s3://your-bucket/poma")  # -> [{"file_id", "content"}, ...]

assemble() auto-detects the result shape (Qdrant among 12 supported stores) and returns deduplicated cheatsheets — the same content-free-index discipline, without PomaQdrant's Qdrant-specific upsert helpers. Use whichever fits: PomaQdrant for a batteries-included Qdrant integration, vektoria when Qdrant is one of several stores you retrieve from.

Feed Qdrant from the parser you already run ​

Pair this page with your ingestion side — the same chunksets, straight from your OCR/parsing tool's raw output:

Running a different store? The same chunk design applies: Azure AI Search · Chroma · Elasticsearch · FAISS · LanceDB · Milvus · MongoDB Atlas · OpenSearch · pgvector · Pinecone · Redis · Turbopuffer · Vespa · Weaviate — or browse all pipeline recipes.

Fundamentals: RAG chunking guide · RAG architecture guide · POMA chunksets.