Source: http://www.poma-ai.com/docs/pipelines/paddleocr-vl-to-weaviate

# The Missing Link Between PaddleOCR-VL and Optimal Retrieval in Weaviate

<ByAuthor />

**The short answer:** PaddleOCR-VL gives you self-hosted, layout-aware OCR behind your own HTTP endpoint; Weaviate gives you built-in hybrid search — BM25F plus vectors, blended by `alpha` — over filterable properties. Wired together naively — flatten, split, embed — they still produce mediocre RAG, and a pipeline that only handles one of PaddleOCR-VL's two accepted shapes can silently drop whole pages from both halves of the hybrid at once. The missing link is POMA: `PrimeCut().ingest()` consumes the raw layout-parsing JSON (auto-detected, either shape), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets), and each becomes one Weaviate object with `file_id`/`page`/`depth` properties and your embedding as its vector.

## What each end of the pipeline actually provides

**PaddleOCR-VL** runs self-hosted — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a `/layout-parsing` HTTP API you operate. It returns either the PaddleX serving envelope (`result.layoutParsingResults[]`, with a `markdown` object plus a `prunedResult` block list) or a raw `save_to_json()` page dict (`parsing_res_list`: flat blocks with `block_label`, `block_content`, `block_id`, `block_order`). Both are the tool's own fingerprints, and a batch job can legitimately emit either one per document. Details: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl).

**Weaviate** provides collections with named vectors, built-in hybrid search (BM25F + vector, blended by `alpha`), filterable properties, cross-references between collections, and generative/reranker modules — open source or Weaviate Cloud. What it doesn't provide: any opinion about what an object should contain. BM25F scores whatever text you stored; the vector index ranks whatever you embedded — nothing, if a page never made it into a collection object at all. Details: [The Optimal Chunks for the Best Retrieval in Weaviate](/optimal-chunks-weaviate).

## The naive wiring, and where it breaks

The common recipe — call `/layout-parsing`, assume every response has `markdown.text`, split, embed, `data.insert(...)` — breaks in a pair-specific way:

- **Handling only one accepted shape drops pages, not just quality.** Code written against the serving envelope's `markdown` object throws or silently skips any page that instead arrives as a bare `save_to_json()` block dict — a real possibility across a self-hosted batch pipeline processing many documents over time. That page is absent from both BM25F and vector search, not degraded in either.
- **`page_index` vanishes at the join**, so the one property Weaviate needs for page filters and page-cited answers holds nothing truthful.
- **Overlap and block-order mistakes pollute both hybrid halves at once.** Splitter overlap duplicates boundary spans in both BM25F and vector rankings; sorting blocks by `block_id` instead of `block_order` scrambles the exact tokens BM25F depends on.

Chunksets fix this at the unit level: both PaddleOCR-VL shapes route through the same hierarchy-preserving assembly, no page silently dropped, no overlap, exact tokens intact. On a reference legal document, assembled cheatsheets delivered 337 tokens of retrieved context versus 1,542 from a recursive character splitter, with zero information loss ([methodology](/document-ingestion-chunking-rag)).

## The pipeline, end to end

```bash
pip install requests poma weaviate-client sentence-transformers
```

```python
import json
import os
import requests
import weaviate
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.init import Auth
from weaviate.classes.query import Filter
from poma import PrimeCut
from sentence_transformers import SentenceTransformer

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("contract.paddleocr-vl.json")  # PaddleOCR-VL shape auto-detected

# 3. One collection of chunksets, hierarchy as filterable properties.
client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)
client.collections.create(
    "Chunkset",
    vectorizer_config=Configure.Vectorizer.none(),  # bring your own vectors
    properties=[
        Property(name="content", data_type=DataType.TEXT),
        Property(name="file_id", data_type=DataType.TEXT),
        Property(name="page", data_type=DataType.INT),
        Property(name="depth", data_type=DataType.INT),
        Property(name="chunkset_index", data_type=DataType.INT),
    ],
)
chunksets = client.collections.get("Chunkset")

model = SentenceTransformer("all-MiniLM-L6-v2")
file_id = result.chunks[0].file_id
for i, cs in enumerate(result.chunksets):
    chunksets.data.insert(
        properties={"content": cs.to_embed, "file_id": file_id,
                    "page": cs.page, "depth": cs.depth, "chunkset_index": i},
        vector=model.encode(cs.to_embed).tolist(),
    )

# 4. Hybrid query: BM25F on content + your vector, blended by alpha.
query = "What are the early termination conditions?"
res = chunksets.query.hybrid(
    query=query,
    vector=model.encode(query).tolist(),
    alpha=0.5,
    filters=Filter.by_property("file_id").equal(file_id),
    limit=3,
)
cheatsheet = "\n\n".join(o.properties["content"] for o in res.objects)
client.close()
```

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), prefers `markdown.text` when present and falls back to `parsing_res_list` sorted by `block_order` when it isn't — for every page, not just the majority shape — drops running furniture, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set `external_ocr_source` to `"paddleocr_vl"` or `"none"`.

## Metadata mapping: POMA fields → Weaviate primitives

| POMA chunk field | Weaviate primitive | What it enables |
| --- | --- | --- |
| `to_embed` | object vector (bring-your-own) | paraphrase retrieval |
| `to_embed` text | `content` TEXT property, BM25F-indexed | exact-term half of hybrid search |
| `file_id` | TEXT property, `Filter.by_property` | scope queries to one document |
| `page` (from PaddleOCR-VL `page_index`, 1-based) | INT property | page filters, page-cited answers |
| `depth` | INT property | filter/re-rank by hierarchy level |
| `chunkset_index` | INT property | stable ordering, independent of `block_id` |
| chunkset lineage | optional cross-reference to a `Chunk` collection | chunk-level provenance without a second store |

## Frequently asked questions

### How do I get self-hosted PaddleOCR-VL results into Weaviate for RAG?

Save the raw `/layout-parsing` JSON, run `PrimeCut().ingest()` on it (auto-detected, hierarchy rebuilt), create a collection with `file_id`/`page`/`depth` properties, insert one object per chunkset with your own vector, and retrieve with `query.hybrid`.

### What happens if a naive pipeline only handles PaddleOCR-VL's markdown shape?

Pages that instead arrive as a raw `save_to_json()` block dict get skipped or crash the pipeline — they never become Weaviate objects, so they're missing from BM25F and vector search alike, not just degraded in one.

### What properties should a Weaviate collection have for PaddleOCR-VL content?

`content` (TEXT, BM25F), `file_id` (TEXT), `page` (INT, from `page_index`), `depth` (INT), `chunkset_index` (INT). POMA emits all of these regardless of which accepted shape a given page arrived in.

### Can I self-host both PaddleOCR-VL and Weaviate for a fully on-premises pipeline?

Yes — both run as open source you operate yourself. Only the layout-parsing result JSON needs to reach POMA's chunking API; documents, embeddings, and the index itself can stay on-premises.

### What hybrid alpha should I use in Weaviate for PaddleOCR-VL content?

Start at `alpha=0.5`. PP-DocLayoutV3 preserves exact tokens worth matching on BM25F — but only if blocks were assembled in `block_order`, not scrambled by sorting on `block_id`.

## Related recipes

Same parser, different store: [PaddleOCR-VL → Qdrant](/pipelines/paddleocr-vl-to-qdrant) · [PaddleOCR-VL → Pinecone](/pipelines/paddleocr-vl-to-pinecone) · [PaddleOCR-VL → Chroma](/pipelines/paddleocr-vl-to-chroma)

Same store, different parser: [Mistral OCR → Weaviate](/pipelines/mistral-ocr-to-weaviate) · [LlamaParse → Weaviate](/pipelines/llamaparse-to-weaviate) · [Azure Document Intelligence → Weaviate](/pipelines/azure-document-intelligence-to-weaviate)

Foundations: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl) · [The Optimal Chunks for Weaviate](/optimal-chunks-weaviate) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)