Source: http://www.poma-ai.com/docs/optimal-chunks-weaviate

# Weaviate Chunking Strategies for RAG: Late Chunking and Chunksets

<ByAuthor />

**The short answer:** the optimal chunks for Weaviate are **chunksets** — self-explanatory units that carry their full heading lineage — stored in one collection with hierarchy metadata (`file_id`, `page`, `depth`) as filterable properties and retrieved with a **hybrid query** (BM25F + vector, blended by `alpha`). Weaviate is one of the few databases where hybrid search is built in rather than bolted on; what decides your retrieval quality is what you put *into* it. Chunksets feed both halves of the hybrid at once: complete thoughts for the embedding, heading terms for the keyword index.

This page covers the Weaviate chunking strategies Weaviate itself teaches — fixed-size, semantic, and late chunking — what each of them leaves unsolved, and the collection and property design that gets the most out of hybrid search.

## Weaviate ships hybrid search out of the box — over whatever you feed it

Weaviate's primitives are a gift to RAG engineers:

- **Built-in hybrid search** — BM25F keyword scoring and vector similarity fused server-side in a single query, weighted by `alpha` (0 = pure keyword, 1 = pure vector).
- **Collections with named vectors** — several embeddings per object when one model isn't enough.
- **Filterable properties** — scope any query by document, tenant, page, or depth without leaving the search.
- **Cross-references** — typed links between collections, so relations like chunkset ↔ member chunks can be modeled as data.
- **Generative and reranker modules** — post-retrieval steps live next to the index instead of in your glue code.

None of these primitives, however, know anything about your documents. Hybrid search fuses two rankings *over whatever objects you inserted*. If those objects are context-free fragments — paragraphs cut loose from the sections that give them meaning — Weaviate will fuse two rankings of context-free fragments, at any scale, with excellent latency. **The ceiling on retrieval quality is set before the first object is inserted.**

## Weaviate chunking strategies, as Weaviate teaches them

Weaviate publishes its own curriculum on this: the Academy course [Document Chunking Strategies](https://academy.weaviate.io/courses/wa921-py) and the blog post [Chunking Strategies for RAG](https://weaviate.io/blog/chunking-strategies-for-rag). Both sort methods by how the boundary is chosen, which is a good map of the field:

- **Fixed-size.** Split at a set token or character count, with an optional overlap window. Weaviate's stated starting point is 512 tokens with 50–100 tokens of overlap, then iterate against your retrieval metrics.
- **Variable-size.** The boundary follows the text. Recursive splitting on separators, document-based splitting on Markdown headings, HTML tags or code functions, and semantic chunking, which cuts where the embedding similarity between adjacent passages drops.
- **Mixed and meaning-aware.** LLM-based chunking, agentic chunking that picks a strategy per document, hierarchical chunking that builds summary levels above the detail, adaptive chunking that varies parameters by content density, and late chunking.

The taxonomy is organised around one question: *where does the cut go?* That is a real question, and Weaviate's answers to it are sound. But none of the three families says what travels **with** the piece once the cut is made. The overlap recommendation is the tell — overlap exists to smuggle a little neighbouring context into a unit that lost it at the boundary. It is a workaround for a unit that cannot stand alone, priced in duplicate vectors and duplicate term statistics, which in a hybrid index you pay for twice.

A fourth option is available and the taxonomy does not name it: keep the document's own hierarchy attached to the piece. Then the cut point stops being load-bearing, because a unit that carries `Master Services Agreement → Termination clauses → Early termination` is interpretable wherever you cut it.

## Weaviate late chunking, and where chunksets fit

[Late chunking](https://weaviate.io/blog/late-chunking) is the most interesting idea in Weaviate's material, and the one most often misread. The mechanism, in order:

1. Feed the **whole document** to a long-context embedding model — Weaviate's post uses `jina-embeddings-v2-small-en` with an 8192-token window — and keep the per-token embeddings instead of pooling immediately.
2. Mean-pool those token embeddings **within each chunk's span**.
3. Index the resulting one-vector-per-chunk output exactly as before.

Because pooling happens after the model has seen the whole document, each chunk vector is conditioned on its surroundings. A chunk that says "she signed it in March" embeds with knowledge of the *Alice* two pages earlier. Weaviate frames the contrast with ColBERT-style late interaction, which preserves the same token-level detail but stores every token embedding and pays for it in index size; late chunking pools back down, so storage matches naive chunking. Weaviate reports it takes under 30 lines of code and needs no change to the retrieval pipeline, and is candid that benchmark data on it is still thin.

Two constraints are worth stating plainly. First, it requires an embedding model you can drive at token level with mean pooling, over a context long enough to hold the document — that rules out several hosted embedding APIs. Second, and more important for RAG: **late chunking fixes the vector, not the text.** The object you stored still reads as a bare fragment. Weaviate's BM25F side indexes that fragment's terms, the LLM receives that fragment in the prompt, and the human checking a citation sees that fragment. Better retrieval, same unreadable evidence.

That is exactly the half a chunkset fixes, and the two compose cleanly. A chunkset's `to_embed` is plain text with no assumptions about how it will be embedded, so you can:

- Use chunkset spans as the **pooling boundaries** for late chunking, rather than arbitrary token windows. The pooled vector then covers a span that is a real unit of the document.
- Store the chunkset text — leaf content plus ancestor headings — in the `content` property, so BM25F matches section vocabulary and the LLM reads a self-explanatory passage.
- Keep `file_id`, `page`, `depth` as filterable properties either way.

Late chunking conditions the embedding; the chunkset conditions everything downstream of it. Neither substitutes for the other.

## What "optimal chunks" means for Weaviate, concretely

Four properties separate a Weaviate collection that answers questions from one that returns trivia:

1. **Every object is self-explanatory.** A chunkset is a root-to-leaf path through the document — `Master Services Agreement → Termination clauses → Early termination → "Either party may…"`. Retrieved alone, it still tells the LLM (and the human reading citations) exactly where it sits. A bare 512-token window does not. See [POMA chunksets](/learn/chunking/chunksets).
2. **Both hybrid halves get fed.** This is Weaviate-specific leverage: the heading breadcrumbs in a chunkset are exact, high-signal terms, so BM25F matches section vocabulary ("Termination", "§ 7.2") that a fragment never contains — while the same self-contained text gives the embedding a complete thought to encode. One unit, two better rankings, one fusion.
3. **No overlap, no near-duplicates.** Overlap is a patch for splitters that cut mid-thought — and in Weaviate it pollutes *two* indexes: near-identical vectors on the dense side, repeated term statistics on the BM25F side. Chunksets carry context structurally, so your `limit=5` returns five *different* candidates.
4. **Hierarchy in filterable properties.** `file_id`, `chunkset_index`, `page`, `depth` as properties turn Weaviate filters into document-aware operations: scope a hybrid query to one contract, exclude appendices by depth, cite pages in the answer.

## The pattern: one chunkset collection, hybrid query, cheatsheets client-side

No plugin needed — the Weaviate v4 Python client and the POMA SDK wire together in a few lines:

```bash
pip install poma weaviate-client
```

```python
import os

import weaviate
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.init import Auth
from weaviate.classes.query import Filter

from poma import PrimeCut
from poma.utils import unpack_poma_archive

# 1. Chunk any document — PDF, DOCX, HTML, or a bring-your-own-OCR result JSON.
#    Keep the .poma archive: page numbers live on its chunk records.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.pdf", download_dir="store", filename="contract.poma")

# Hierarchy lookups by chunk index. The SDK exposes depth; page comes from
# the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
# -1 when the source has no pages (Markdown, text)
page_of = {c["chunk_index"]: (c.get("page") if c.get("page") is not None else -1) for c in archive["chunks"]}

# 2. One collection of chunksets; hierarchy fields as filterable properties.
wv = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)  # or weaviate.connect_to_local() for a local instance

chunksets = wv.collections.create(
    name="Chunkset",
    # bring your own vectors (python-client < 4.16: vectorizer_config=Configure.Vectorizer.none())
    vector_config=Configure.Vectors.self_provided(),
    properties=[
        Property(name="content", data_type=DataType.TEXT),  # BM25F searches this
        Property(name="file_id", data_type=DataType.TEXT),
        Property(name="page", data_type=DataType.INT),
        Property(name="depth", data_type=DataType.INT),
        Property(name="chunkset_index", data_type=DataType.INT),
    ],
)

for cs, vec in zip(result.chunksets, embed([c.to_embed for c in result.chunksets])):
    leaf = cs.chunks[-1]  # chunk indices in document order; the last is the leaf
    chunksets.data.insert(
        properties={
            "content": cs.to_embed,
            "file_id": cs.file_id,
            "chunkset_index": cs.chunkset_index,
            "page": page_of[leaf],
            "depth": depth_of[leaf],
        },
        vector=vec,
    )

# 3. Hybrid query: BM25F + vector fused by alpha, scoped to one document.
question = "What does the contract say about early termination?"
response = chunksets.query.hybrid(
    query=question,                    # feeds the BM25F side
    vector=embed([question])[0],       # feeds the vector side
    alpha=0.5,                         # 0 = pure keyword, 1 = pure vector
    limit=5,
    filters=Filter.by_property("file_id").equal(result.chunksets[0].file_id),
)

# 4. Assemble a cheatsheet client-side: retrieved chunksets share ancestors,
#    so deduplicate the common heading lineage and merge into one context block.
hits = [obj.properties["content"] for obj in response.objects]
```

`embed(...)` is your embedding model of choice; if you prefer, configure a Weaviate vectorizer module instead of `Vectorizer.none()` and let the cluster embed `content` for you — the collection design stays the same.

Step 4 is where the token savings land: retrieved chunksets from the same section repeat the same breadcrumbs, and merging them deduplicates the lineage into one coherent block — POMA calls this a **cheatsheet**. On one reference legal document that meant **337 tokens** of context versus **1,542** for a recursive-splitter baseline, with nothing lost. One document is an illustration, not a benchmark; the [ingestion guide](/document-ingestion-chunking-rag) has the broader numbers.

## Do you need cross-references?

Weaviate can model the chunk ↔ chunkset relation explicitly: a `Chunkset` collection cross-referencing a `Chunk` collection lets you traverse from any retrieved unit to its exact members for auditing, highlighting, or re-chunking. It is a genuinely nice capability — and for the hot retrieval path you usually don't need it, because the chunkset's text already *contains* its lineage. Keep retrieval a single hybrid query on one flat collection; add cross-references when lineage must be traversable as data, not just readable as text. The `.poma` archive remains the portable source of truth either way.

## Chunk shapes in Weaviate, compared

| Property | Fixed 512-token chunks + overlap | Semantic chunks | **POMA chunksets** |
| --- | --- | --- | --- |
| Object is self-explanatory when retrieved alone | ✗ | Sometimes | **✓ (by construction)** |
| BM25F side sees section vocabulary | ✗ (fragment terms only) | Sometimes | **✓ (heading breadcrumbs)** |
| Near-duplicates polluting both hybrid rankings | Many (overlap) | Few | **None** |
| Page + hierarchy as filterable properties | Manual | Manual | **✓ (`file_id`, `page`, `depth`)** |
| Tables survive embedding | ✗ (cut mid-row) | Fragile | **✓ (HTML, never cut)** |
| Post-retrieval assembly | Concatenate raw hits | Concatenate raw hits | **Cheatsheets (deduped lineage)** |

## Frequently asked questions

### What is the optimal chunk size for Weaviate?

There is no universal token count — the optimal unit is a chunk that is self-explanatory without its neighbors. Chunksets achieve that at any size and feed both hybrid halves: dense meaning for the vector side, heading terms for the BM25F side. If you must fix a size, start at 512 tokens — but the size knob cannot restore missing context.

### What should I embed in Weaviate for RAG?

Embed the chunkset's `to_embed` text — leaf content plus ancestor headings — and store the same text in a `TEXT` property so BM25F can search it. Add `file_id`, `chunkset_index`, `page`, `depth` as filterable properties — `page` comes from the chunk records in the `.poma` archive (the chunkset’s leaf chunk); the SDK objects expose `depth` but not `page`. The vector, the keyword index, and the filters then agree on what a unit of meaning is.

### What alpha should I use for Weaviate hybrid search?

`alpha` blends the two rankings: 0 is pure BM25F, 1 is pure vector, 0.5 weights them equally. Corpora full of exact tokens (statute numbers, SKUs, defined terms) deserve meaningful keyword weight. Chunksets improve both sides simultaneously, so tuning alpha becomes a preference, not a rescue operation.

### Should I use cross-references to model chunk hierarchy in Weaviate?

Only when lineage must be traversable as data — auditing, highlighting, re-chunking. For retrieval itself, the chunkset's text already contains its lineage, so a single hybrid query on one flat collection is faster and simpler than a graph hop.

### How does chunk overlap affect Weaviate hybrid search?

It pollutes both indexes: near-identical vectors on the dense side, repeated terms on the BM25F side — so after fusion, top-k is even more redundant than in a pure vector store. Chunks that carry hierarchy don't need overlap.

### Does POMA integrate with Weaviate?

Yes — via the standard SDK pattern shown above: `PrimeCut().ingest`, embed each chunkset's `to_embed`, insert into a collection with hierarchy properties, `collection.query.hybrid(...)`, then assemble cheatsheets client-side. No separate plugin required.

### What is late chunking in Weaviate?

Late chunking inverts the usual order: instead of splitting a document and embedding each piece alone, you embed the whole document with a long-context model, then mean-pool the resulting token embeddings over each chunk's span. Every chunk vector is therefore conditioned on the full document, so a pronoun in a later chunk still points at the name introduced earlier. Weaviate contrasts it with ColBERT-style late interaction, which keeps every token embedding and pays for it in storage; late chunking pools down to one vector per chunk, so index size matches naive chunking. It needs a long-context embedding model whose token embeddings and mean pooling you can reach, and Weaviate notes that public benchmark data is still limited.

### Do late chunking and chunksets work together?

Yes, and they fix different halves of the problem. Late chunking conditions the vector on the surrounding document but leaves the stored text a bare fragment, so the passage the LLM reads and the citation a human checks are as context-free as before. A chunkset fixes the text: the heading lineage is part of the string. Because a chunkset's `to_embed` field is plain text with no assumptions about the embedding method, you can use chunkset boundaries as the pooling spans for late chunking and store the same chunkset text in the `TEXT` property that BM25F searches.

### Is semantic chunking the right strategy for Weaviate?

Semantic chunking cuts where embedding similarity between adjacent passages drops, which places boundaries better than a fixed token count and is a fair default when documents have no usable structure. It is still only a decision about where to cut. The resulting passage carries no record of the section it came from, so Weaviate's BM25F half never sees the section vocabulary and the LLM never sees the breadcrumb. When the document has headings, splitting along the existing hierarchy and keeping the root-to-leaf path in the text gives both halves of hybrid search more to work with than a better-placed cut does.

## Retrieval with vektoria's assemble()

POMA's `poma.vektoria` package keeps the collection content-free: Weaviate stores only `(id, vector, {file_id, chunkset_index})`, while the actual chunk content lives on a **volume** (a path, `s3://`, or `gs://` URL). After your normal Weaviate query, hand the raw result to `assemble()`:

```python
from poma.vektoria import assemble
from weaviate.classes.query import MetadataQuery

res = coll.query.near_vector(qv, limit=10, return_metadata=MetadataQuery(distance=True))
context = assemble(res, volume="s3://your-bucket/poma")  # -> [{"file_id", "content"}, ...]
```

`assemble()` auto-detects Weaviate's result shape (one of 12 supported stores) and returns deduplicated cheatsheets. Properties (including `file_id`/`chunkset_index`) come back automatically in Weaviate v4 — `return_metadata=MetadataQuery(distance=True)` is only needed for the distance score itself.

## Feed Weaviate from the parser you already run

Pair this page with your ingestion side — the same chunksets, straight from your OCR/parsing tool's raw output:

- [AWS Textract → Weaviate](/pipelines/textract-to-weaviate)
- [Azure Document Intelligence → Weaviate](/pipelines/azure-document-intelligence-to-weaviate)
- [Docling → Weaviate](/pipelines/docling-to-weaviate)
- [LlamaParse → Weaviate](/pipelines/llamaparse-to-weaviate)
- [Marker → Weaviate](/pipelines/marker-to-weaviate)
- [Mistral OCR → Weaviate](/pipelines/mistral-ocr-to-weaviate)
- [PaddleOCR-VL → Weaviate](/pipelines/paddleocr-vl-to-weaviate)
- [Unstructured.io → Weaviate](/pipelines/unstructured-to-weaviate)

Running a different store? The same chunk design applies: [Azure AI Search](/optimal-chunks-azure-ai-search) · [Chroma](/optimal-chunks-chroma) · [Elasticsearch](/optimal-chunks-elasticsearch) · [FAISS](/optimal-chunks-faiss) · [LanceDB](/optimal-chunks-lancedb) · [Milvus](/optimal-chunks-milvus) · [MongoDB Atlas](/optimal-chunks-mongodb-atlas) · [OpenSearch](/optimal-chunks-opensearch) · [pgvector](/optimal-chunks-pgvector) · [Pinecone](/optimal-chunks-pinecone) · [Qdrant](/optimal-chunks-qdrant) · [Redis](/optimal-chunks-redis) · [Turbopuffer](/optimal-chunks-turbopuffer) · [Vespa](/optimal-chunks-vespa) — or browse [all pipeline recipes](/pipelines/).

Fundamentals: [RAG chunking guide](/guides/rag-chunking/) · [RAG architecture guide](/guides/rag-architecture/) · [POMA chunksets](/learn/chunking/chunksets).