Source: http://www.poma-ai.com/docs/pipelines/paddleocr-vl-to-chroma

# The Missing Link Between PaddleOCR-VL and Optimal Retrieval in Chroma

<ByAuthor />

**The short answer:** PaddleOCR-VL gives you self-hosted, layout-aware OCR behind your own HTTP endpoint; Chroma gives you the fastest path from zero to working retrieval — an embedded, persistent vector store with metadata filtering and pluggable embedding functions. Wired together naively — flatten, split, embed — the prototype works just well enough to ship its worst decision everywhere, and inconsistent handling of PaddleOCR-VL's two accepted shapes across batch workers compounds it with a metadata schema that's incomplete for a silent subset of documents. The missing link is POMA: `PrimeCut().ingest()` consumes the raw layout-parsing JSON (auto-detected, either shape, every time), emits hierarchy-preserving [chunksets](/learn/chunking/chunksets), and each lands in Chroma with `file_id`, `page`, `depth`, and `chunk_index` metadata ready for `where` filters — with the `.poma` archive as a portable source of truth when the prototype graduates.

## What each end of the pipeline actually provides

**PaddleOCR-VL** runs self-hosted — PP-DocLayoutV3 for layout detection, PaddleOCR-VL served via vLLM for reading, behind a `/layout-parsing` HTTP API you operate. It returns either the PaddleX serving envelope (`result.layoutParsingResults[]`, with a `markdown` object plus a `prunedResult` block list) or a raw `save_to_json()` page dict (`parsing_res_list`: flat blocks with `block_label`, `block_content`, `block_id`, `block_order`) — both legitimate, both liable to appear across a multi-worker batch pipeline. Details: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl).

**Chroma** provides a local-first store that runs embedded or client/server, collections with default or bring-your-own embedding functions, `where` filters over metadata, and `where_document` filters over content. What it doesn't provide: any opinion about what a document entry should contain, or metadata it wasn't given consistently. `where={"file_id": ...}` matches nothing for an entry that never received one. Details: [The Optimal Chunks for the Best Retrieval in Chroma](/optimal-chunks-chroma).

## The naive wiring, and where it breaks

The common recipe — call `/layout-parsing`, hand-roll a block-to-text converter, split, `collection.add(documents=windows)` — breaks in a pair-specific way:

- **Inconsistent shape handling ships an inconsistent schema.** A self-hosted PaddleOCR-VL deployment commonly runs several batch workers; if one script only expects the serving envelope's `markdown` object and another only expects raw `parsing_res_list` dicts, the Chroma collection ends up with some entries carrying a `page` integer and others carrying none — `where` filters then silently drop the incomplete subset with no error raised.
- **The prototype decision gets baked in.** Chroma is where RAG pipelines are born; the chunking chosen in hour one is rarely revisited, so a hand-rolled flattener's gaps travel unnoticed into the production database it eventually ports to.
- **Retrieved fragments arrive without lineage**, so the LLM answers out of context — a failure that looks like "Chroma isn't good enough" when the actual defect is upstream, in how the two PaddleOCR-VL shapes were merged.

POMA's chunksets fix this at the source: one chunker, both shapes, the same hierarchy metadata every time. On a notoriously hard reference legal document, chunksets plus cheatsheet assembly delivered the answer in 337 tokens of retrieved context versus 1,542 for a recursive character splitter, with zero information loss ([methodology](/document-ingestion-chunking-rag)).

## The pipeline, end to end

```bash
pip install requests poma chromadb
```

```python
import json
import requests
import chromadb
from poma import PrimeCut

# 1. Your existing self-hosted call — unchanged. `/layout-parsing` is the
#    PaddleX-compatible endpoint your layout-server + PaddleOCR-VL stack exposes.
resp = requests.post(
    "http://layout-server:40111/layout-parsing",
    json={"file": "<base64-encoded PDF>", "fileType": 0},
)
resp.raise_for_status()
with open("contract.paddleocr-vl.json", "w") as f:
    json.dump(resp.json(), f)

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("contract.paddleocr-vl.json")  # PaddleOCR-VL shape auto-detected

# 3. Chroma — persistent local store; the collection's embedding
#    function (default or your own) embeds the documents on add.
chroma = chromadb.PersistentClient(path="./chroma")
collection = chroma.get_or_create_collection(name="contracts")

# 4. Add chunksets — documents, stable IDs, where-filterable metadata.
collection.add(
    ids=[f"{cs.file_id}:{cs.chunk_index}" for cs in result.chunksets],
    documents=[cs.to_embed for cs in result.chunksets],
    metadatas=[
        {
            "file_id": cs.file_id,
            "page": cs.page,
            "depth": cs.depth,
            "chunk_index": cs.chunk_index,
        }
        for cs in result.chunksets
    ],
)

# 5. Query with a metadata filter, then assemble the cheatsheet.
res = collection.query(
    query_texts=["What are the early termination conditions?"],
    n_results=5,
    where={"file_id": "contract.paddleocr-vl.json"},
)
pairs = sorted(
    zip(res["documents"][0], res["metadatas"][0]),
    key=lambda p: p[1]["chunk_index"],
)
cheatsheet = "\n\n".join(dict.fromkeys(doc for doc, _ in pairs))
```

POMA validates the payload shape up front (a corrupted or mislabeled upload 422s immediately), prefers `markdown.text` when present and falls back to `parsing_res_list` sorted by `block_order` when it isn't — the same logic for every worker, every document — drops running furniture, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set `external_ocr_source` to `"paddleocr_vl"` or `"none"`.

## Prototype today, port tomorrow

The `.poma` archive that `PrimeCut` produces is the portable source of truth for the document — chunks, chunksets, and hierarchy metadata, independent of any database and independent of which of PaddleOCR-VL's two shapes fed it. Loading it into Chroma is one loop; loading the same archive into Qdrant, Weaviate, or Pinecone is the same loop against a different client, because `file_id`/`page`/`depth` map onto every store's filtering primitive. The prototype and the production system disagree only about infrastructure, never about content — no re-running the self-hosted OCR call, no re-chunk, no drift between what the demo retrieved and what production retrieves.

## Metadata mapping: POMA fields → Chroma primitives

| POMA chunk field | Chroma primitive | What it enables |
| --- | --- | --- |
| `to_embed` | `documents` entry, embedded by the collection's embedding function | semantic retrieval; `where_document` content filters |
| `file_id` | `metadatas` key, `where={"file_id": ...}` | scope queries to one document |
| `page` (from PaddleOCR-VL `page_index`, 1-based) | `metadatas` key, `where` with range operators | page-cited answers, page-range filters |
| `depth` | `metadatas` key | filter/re-rank by hierarchy level |
| `chunk_index` | `metadatas` key + part of the entry `id` | stable IDs, ordering, independent of `block_id` |
| chunkset lineage | `.poma` archive (portable source of truth) | identical reload into any other vector DB |

## Frequently asked questions

### How do I get self-hosted PaddleOCR-VL results into Chroma for RAG?

Save the raw `/layout-parsing` JSON, run `PrimeCut().ingest()` on it (auto-detected, hierarchy rebuilt), then `collection.add()` with each chunkset's `to_embed` as the document and `file_id`/`page`/`depth`/`chunk_index` metadata. Query with `where` filters and assemble cheatsheets client-side.

### What goes wrong when different workers process PaddleOCR-VL's two shapes inconsistently?

If one worker's script only handles the serving envelope and another only handles raw `save_to_json` dicts, the resulting collection has a schema gap — some entries lack `page` — and `where` filters silently exclude that subset without an error.

### What metadata should Chroma store for PaddleOCR-VL chunks so where filters work?

`file_id`, `page` (from `page_index`), `depth`, and `chunk_index`. POMA writes all four consistently no matter which of the two accepted shapes produced a given chunk.

### Will a Chroma prototype built on self-hosted PaddleOCR-VL chunks port to a production vector database?

The `.poma` archive reloads into Qdrant, Weaviate, or Pinecone without re-running OCR or re-chunking. A hand-rolled flattener's schema gaps also port — which is why routing both shapes through one chunker from day one matters.

### Does staying self-hosted through Chroma matter for data locality?

Yes — PaddleOCR-VL and Chroma can both run entirely on infrastructure you control. Only the layout-parsing result JSON needs to reach POMA's chunking API.

## Related recipes

Same parser, different store: [PaddleOCR-VL → Qdrant](/pipelines/paddleocr-vl-to-qdrant) · [PaddleOCR-VL → Pinecone](/pipelines/paddleocr-vl-to-pinecone) · [PaddleOCR-VL → Weaviate](/pipelines/paddleocr-vl-to-weaviate)

Same store, different parser: [Mistral OCR → Chroma](/pipelines/mistral-ocr-to-chroma) · [LlamaParse → Chroma](/pipelines/llamaparse-to-chroma) · [Docling → Chroma](/pipelines/docling-to-chroma)

Foundations: [The Optimal Chunker for PaddleOCR-VL](/optimal-chunker-paddleocr-vl) · [The Optimal Chunks for Chroma](/optimal-chunks-chroma) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)