Source: http://www.poma-ai.com/docs/pipelines/paddleocr-vl-to-weaviate

# PaddleOCR-VL to Weaviate: Chunking for RAG

<ByAuthor />

**The short answer:** PaddleOCR-VL gives you self-hosted, layout-aware OCR behind your own HTTP endpoint; Weaviate gives you built-in hybrid search — BM25F plus vectors, blended by `alpha` — over filterable properties. Wired together naively — flatten, split, embed — they still produce mediocre RAG, and a pipeline that only handles one of PaddleOCR-VL's two accepted shapes can silently drop whole pages from both halves of the hybrid at once. The missing link is POMA: `PrimeCut().ingest()` consumes the raw layout-parsing JSON (auto-detected, either shape), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets), and each becomes one Weaviate object with `file_id`/`page`/`depth` properties and your embedding as its vector.

## What each end of the pipeline actually provides

**PaddleOCR-VL** runs self-hosted — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a `/layout-parsing` HTTP API you operate. It returns either the PaddleX serving envelope (`result.layoutParsingResults[]`, with a `markdown` object plus a `prunedResult` block list) or a raw `save_to_json()` page dict (`parsing_res_list`: flat blocks with `block_label`, `block_content`, `block_id`, `block_order`). Both are the tool's own fingerprints, and a batch job can legitimately emit either one per document. Details: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl).

**Weaviate** provides collections with named vectors, built-in hybrid search (BM25F + vector, blended by `alpha`), filterable properties, cross-references between collections, and generative/reranker modules — open source or Weaviate Cloud. What it doesn't provide: any opinion about what an object should contain. BM25F scores whatever text you stored; the vector index ranks whatever you embedded — nothing, if a page never made it into a collection object at all. Details: [The Optimal Chunks for the Best Retrieval in Weaviate](/optimal-chunks-weaviate).

## The naive wiring, and where it breaks

The common recipe — call `/layout-parsing`, assume every response has `markdown.text`, split, embed, `data.insert(...)` — breaks in a pair-specific way:

- **Handling only one accepted shape drops pages, not just quality.** Code written against the serving envelope's `markdown` object throws or silently skips any page that instead arrives as a bare `save_to_json()` block dict — a real possibility across a self-hosted batch pipeline processing many documents over time. That page is absent from both BM25F and vector search, not degraded in either.
- **`page_index` vanishes at the join**, so the one property Weaviate needs for page filters and page-cited answers holds nothing truthful.
- **Overlap and block-order mistakes pollute both hybrid halves at once.** Splitter overlap duplicates boundary spans in both BM25F and vector rankings; sorting blocks by `block_id` instead of `block_order` scrambles the exact tokens BM25F depends on.

Chunksets fix this at the unit level: both PaddleOCR-VL shapes route through the same hierarchy-preserving assembly, no page silently dropped, no overlap, exact tokens intact. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss ([methodology](/document-ingestion-chunking-rag)).

## The pipeline, end to end

```bash
pip install requests poma weaviate-client sentence-transformers
```

```python
import json
import os
import requests
import weaviate
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.init import Auth
from weaviate.classes.query import Filter
from poma import PrimeCut
from poma.utils import unpack_poma_archive
from sentence_transformers import SentenceTransformer

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
# keep the .poma archive: page numbers live on its chunk records, not on the SDK objects
result = poma.ingest(
    "contract.paddleocr-vl.json",  # PaddleOCR-VL shape auto-detected
    download_dir="store",
    filename="contract.poma",
)

# Chunk lookups: depth from the SDK objects, page from the archive's chunk records.
depth_of = {c.chunk_index: c.depth for c in result.chunks}
page_of = {
    c["chunk_index"]: c.get("page")
    for c in unpack_poma_archive(poma_archive_path="store/contract.poma")["chunks"]
}

# 3. One collection of chunksets, hierarchy as filterable properties.
client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)
client.collections.create(
    "Chunkset",
    vectorizer_config=Configure.Vectorizer.none(),  # bring your own vectors
    properties=[
        Property(name="content", data_type=DataType.TEXT),
        Property(name="file_id", data_type=DataType.TEXT),
        Property(name="page", data_type=DataType.INT),
        Property(name="depth", data_type=DataType.INT),
        Property(name="chunkset_index", data_type=DataType.INT),
    ],
)
chunksets = client.collections.get("Chunkset")

model = SentenceTransformer("all-MiniLM-L6-v2")
file_id = result.chunks[0].file_id
for cs in result.chunksets:
    leaf = cs.chunks[-1]  # document order; the last chunk index is the leaf
    chunksets.data.insert(
        properties={"content": cs.to_embed, "file_id": file_id,
                    "page": page_of[leaf], "depth": depth_of[leaf],
                    "chunkset_index": cs.chunkset_index},
        vector=model.encode(cs.to_embed).tolist(),
    )

# 4. Hybrid query: BM25F on content + your vector, blended by alpha.
query = "What are the early termination conditions?"
res = chunksets.query.hybrid(
    query=query,
    vector=model.encode(query).tolist(),
    alpha=0.5,
    filters=Filter.by_property("file_id").equal(file_id),
    limit=3,
)
cheatsheet = "\n\n".join(o.properties["content"] for o in res.objects)
client.close()
```

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), prefers `markdown.text` when present and falls back to `parsing_res_list` sorted by `block_order` when it isn't — for every page, not just the majority shape — drops running furniture, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set `external_ocr_source` to `"paddleocr_vl"` or `"none"`.

## Metadata mapping: POMA fields → Weaviate primitives

| POMA chunk field | Weaviate primitive | What it enables |
| --- | --- | --- |
| `to_embed` | object vector (bring-your-own) | paraphrase retrieval |
| `to_embed` text | `content` TEXT property, BM25F-indexed | exact-term half of hybrid search |
| `file_id` | TEXT property, `Filter.by_property` | scope queries to one document |
| `page` (on the archive's chunk records, from PaddleOCR-VL `page_index`, 1-based; take the chunkset's leaf chunk) | INT property | page filters, page-cited answers |
| `depth` (of the chunkset's leaf chunk) | INT property | filter/re-rank by hierarchy level |
| `chunkset_index` | INT property | stable ordering, independent of `block_id` |
| chunkset lineage | optional cross-reference to a `Chunk` collection | chunk-level provenance without a second store |

## Frequently asked questions

### How do I get self-hosted PaddleOCR-VL results into Weaviate for RAG?

Save the raw `/layout-parsing` JSON, run `PrimeCut().ingest()` on it (auto-detected, hierarchy rebuilt), create a collection with `file_id`/`page`/`depth` properties, insert one object per chunkset with your own vector, and retrieve with `query.hybrid`. `page` comes from the archive's chunk records (the chunkset's leaf chunk) — the SDK objects expose `depth` but not `page`.

### What happens if a naive pipeline only handles PaddleOCR-VL's markdown shape?

Pages that instead arrive as a raw `save_to_json()` block dict get skipped or crash the pipeline — they never become Weaviate objects, so they're missing from BM25F and vector search alike, not just degraded in one.

### What properties should a Weaviate collection have for PaddleOCR-VL content?

`content` (TEXT, BM25F), `file_id` (TEXT), `page` (INT, from `page_index`, read off the archive's chunk records), `depth` (INT), `chunkset_index` (INT). POMA emits all of these regardless of which accepted shape a given page arrived in.

### Can I self-host both PaddleOCR-VL and Weaviate for a fully on-premises pipeline?

Yes — both run as open source you operate yourself. Only the layout-parsing result JSON needs to reach POMA's chunking API; documents, embeddings, and the index itself can stay on-premises.

### What hybrid alpha should I use in Weaviate for PaddleOCR-VL content?

Start at `alpha=0.5`. PP-DocLayoutV3 preserves exact tokens worth matching on BM25F — but only if blocks were assembled in `block_order`, not scrambled by sorting on `block_id`.

## Related recipes

Same parser, different store: [PaddleOCR-VL → Qdrant](/pipelines/paddleocr-vl-to-qdrant) · [PaddleOCR-VL → Pinecone](/pipelines/paddleocr-vl-to-pinecone) · [PaddleOCR-VL → Chroma](/pipelines/paddleocr-vl-to-chroma)

Same store, different parser: [Mistral OCR → Weaviate](/pipelines/mistral-ocr-to-weaviate) · [LlamaParse → Weaviate](/pipelines/llamaparse-to-weaviate) · [Azure Document Intelligence → Weaviate](/pipelines/azure-document-intelligence-to-weaviate)

Foundations: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl) · [The Optimal Chunks for Weaviate](/optimal-chunks-weaviate) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)