Source: http://www.poma-ai.com/docs/pipelines/paddleocr-vl-to-milvus

# The Missing Link Between PaddleOCR-VL and Optimal Retrieval in Milvus

<ByAuthor />

**The short answer:** PaddleOCR-VL gives you self-hosted, layout-labeled document parsing; Milvus gives you filtered vector search at serious scale — partitions, scalar filter expressions, and your pick of HNSW, IVF, or DiskANN. Wired together naively — flatten the labeled blocks, split, embed, insert — the pipeline underuses both, because nothing in between rebuilds the document hierarchy or populates the scalar fields Milvus filters on. The missing link is POMA: `PrimeCut().ingest()` consumes the raw layout-parsing JSON (auto-detected, either PaddleOCR-VL shape), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets), and each chunkset lands in Milvus with `file_id`, `page`, `depth`, and `chunk_index` as scalar fields ready for filter expressions and partition-key routing.

## What each end of the pipeline actually provides

**PaddleOCR-VL** runs entirely on your own infrastructure — a layout-detection model (PP-DocLayoutV3) plus a vision-language OCR model served via vLLM behind a PaddleX-compatible `/layout-parsing` endpoint. It returns a serving envelope with a pre-assembled `markdown` object, or the raw `save_to_json()` block list (`parsing_res_list`) of `{block_label, block_content, block_order}` entries. What it doesn't provide: a link between a heading on one page and its ancestor on another, or any opinion about what a retrieval unit should be. Details: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl).

**Milvus** provides collections with typed schemas, partitions with automatic partition-key routing for multi-tenancy, scalar fields queried through filter expressions (`file_id == "..."`, `page <= 10`), index types from in-memory HNSW through IVF to disk-resident DiskANN, and — in recent versions — sparse vectors with built-in BM25 for hybrid search. What it doesn't provide: any opinion about what a row should contain. Empty scalar fields filter nothing; fragment vectors retrieve fragments. Details: [The Optimal Chunks for the Best Retrieval in Milvus](/optimal-chunks-milvus).

## The naive wiring, and where it breaks

The common recipe — concatenate `parsing_res_list` block content (or `markdown.text` pages) → `RecursiveCharacterTextSplitter` → embed → `client.insert(...)` — breaks in a pair-specific way:

- **`page_index` is `None` for a bare single image, not a PDF page.** A naive pipeline that inserts it directly into Milvus's INT64 `page` field either fails the insert on a non-nullable column, or the client coerces `None` to `0` — which then collides with legitimate index-0 content and quietly corrupts every `page <= N` filter expression built for page-cited answers.
- **The scalar fields end up empty otherwise.** A flat splitter emits anonymous text windows: no page (the join across pages discarded `page_index`), no depth (PP-DocLayoutV3's `doc_title`/`paragraph_title` labels were collapsed to plain text), often no reliable document ID. Milvus's headline feature, filtered search, has nothing to filter on.
- **Overlap bloats the index.** Splitter overlap embeds every boundary span twice, inflating vector count across the board — and it hits DiskANN hardest, where a disk-resident index pays for near-duplicate vectors in build time and disk footprint.

POMA's chunksets fix all three at the source: `page_index` is normalized to a 1-based page number (or left unset, never coerced to a colliding zero), hierarchy metadata comes from PP-DocLayoutV3's own labels, and there's no overlap. On a notoriously hard reference legal document, chunksets plus cheatsheet assembly delivered the answer in 337 tokens of retrieved context versus 1,542 for a recursive character splitter, with zero information loss ([methodology](/document-ingestion-chunking-rag)).

## The pipeline, end to end

```bash
pip install poma requests pymilvus sentence-transformers
```

```python
import json
import os
import requests
from poma import PrimeCut
from pymilvus import DataType, MilvusClient
from sentence_transformers import SentenceTransformer

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("contract.paddleocr-vl.json")  # PaddleOCR-VL shape auto-detected

# 3. Milvus — a schema whose scalar fields mirror the hierarchy metadata.
client = MilvusClient(uri=os.environ["MILVUS_URI"])
schema = MilvusClient.create_schema()
schema.add_field("pk", DataType.INT64, is_primary=True, auto_id=True)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=384)
schema.add_field("text", DataType.VARCHAR, max_length=65535)
schema.add_field("file_id", DataType.VARCHAR, max_length=256, is_partition_key=True)
schema.add_field("page", DataType.INT64)
schema.add_field("depth", DataType.INT64)
schema.add_field("chunk_index", DataType.INT64)

index_params = client.prepare_index_params()
index_params.add_index(field_name="vector", index_type="HNSW", metric_type="COSINE")
client.create_collection("contracts", schema=schema, index_params=index_params)

# 4. Embed chunksets and insert — hierarchy metadata rides along as scalars.
#    cs.page is None for a bare single image; coerce explicitly rather than
#    letting it collide with real page-0 content.
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
client.insert(
    collection_name="contracts",
    data=[
        {
            "vector": model.encode(cs.to_embed).tolist(),
            "text": cs.to_embed,
            "file_id": cs.file_id,
            "page": cs.page if cs.page is not None else -1,
            "depth": cs.depth,
            "chunk_index": cs.chunk_index,
        }
        for cs in result.chunksets
    ],
)

# 5. Filtered search — scope to one document, then assemble the cheatsheet.
query = "What are the early termination conditions?"
hits = client.search(
    collection_name="contracts",
    data=[model.encode(query).tolist()],
    limit=5,
    filter='file_id == "contract.paddleocr-vl.json"',
    output_fields=["text", "page", "depth", "chunk_index"],
)
ordered = sorted(hits[0], key=lambda h: h["entity"]["chunk_index"])
cheatsheet = "\n\n".join(dict.fromkeys(h["entity"]["text"] for h in ordered))
```

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), handles both PaddleOCR-VL shapes, assembles blocks by `block_order` rather than array position, drops running furniture (`header`/`footer`/`page_number`), and normalizes `page_index` to 1-based page numbers before chunking. To force or suppress detection, set `external_ocr_source` to `"paddleocr_vl"` or `"none"`. The final step is the cheatsheet: retrieved chunksets are deduplicated (shared ancestor headings appear once) and merged into a single prompt-ready block — Milvus returns the rows, your client assembles them.

## Metadata mapping: POMA fields → Milvus primitives

| POMA chunk field | Milvus primitive | What it enables |
| --- | --- | --- |
| `to_embed` | `FLOAT_VECTOR` field (add a sparse/BM25 field for hybrid) | paraphrase + exact-term retrieval |
| `file_id` | `VARCHAR` scalar field, **partition key** | per-document filters, automatic multi-tenant routing |
| `page` (PaddleOCR-VL's `page_index`, 0-based, normalized to 1-based; `None` for a bare image) | `INT64` scalar field | page-cited answers, `page <= N` filter expressions |
| `depth` (from `doc_title`/`paragraph_title` label levels) | `INT64` scalar field | filter/re-rank by hierarchy level |
| `chunk_index` (assigned by `block_order`) | `INT64` scalar field | stable, correctly-sequenced ordering at assembly |
| chunkset text | `VARCHAR` field in `output_fields` | assembly without a second store |

## Frequently asked questions

### How do I get self-hosted PaddleOCR-VL results into Milvus for RAG?

Save the raw `/layout-parsing` response, run `PrimeCut().ingest()` on it (auto-detected, hierarchy rebuilt), embed each chunkset's `to_embed`, and insert rows whose scalar fields carry `file_id`, `page`, `depth`, and `chunk_index`. Retrieve with a filtered search and assemble cheatsheets client-side.

### Why does PaddleOCR-VL's page_index need special handling before it reaches Milvus?

`page_index` is `None` for a bare single image. Inserted directly, it either fails a non-nullable INT64 field or coerces to `0`, colliding with real page-0 content in `page <= N` filters. POMA normalizes it to a 1-based number, or leaves it explicitly unset.

### What scalar fields should a Milvus collection have for PaddleOCR-VL chunks?

`file_id` (VARCHAR, the partition-key candidate), `page` (INT64, normalized from `page_index`), `depth` (INT64, from PP-DocLayoutV3's title labels), and `chunk_index` (INT64, assigned by `block_order`) alongside the vector and text.

### Should I use a Milvus partition key for multi-tenant self-hosted PaddleOCR-VL content?

Yes — declare `file_id` as the partition key so filtered searches only touch matching partitions. POMA populates `file_id` on every chunkset from either PaddleOCR-VL output shape, so the key always has a value.

### Does Milvus hybrid sparse plus BM25 search help with self-hosted PaddleOCR-VL documents?

Yes — self-hosted PaddleOCR-VL pipelines often handle documents dense with exact tokens (part codes, clause numbers) that dense embeddings blur. Recent Milvus versions store sparse BM25 vectors next to dense ones and fuse both at query time.

## Related recipes

Same parser, different store: [PaddleOCR-VL → pgvector](/pipelines/paddleocr-vl-to-pgvector) · [PaddleOCR-VL → Turbopuffer](/pipelines/paddleocr-vl-to-turbopuffer) · [PaddleOCR-VL → Vespa](/pipelines/paddleocr-vl-to-vespa)

Same store, different parser: [Mistral OCR → Milvus](/pipelines/mistral-ocr-to-milvus) · [Unstructured.io → Milvus](/pipelines/unstructured-to-milvus) · [Azure Document Intelligence → Milvus](/pipelines/azure-document-intelligence-to-milvus)

Foundations: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl) · [The Optimal Chunks for Milvus](/optimal-chunks-milvus) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)