Source: http://www.poma-ai.com/docs/pipelines/marker-to-weaviate

# The Missing Link Between Marker and Optimal Retrieval in Weaviate

<ByAuthor />

**The short answer:** Marker gives you fast local PDF parsing with real recovered structure; Weaviate gives you hybrid search — BM25F keyword scoring and vector similarity fused by an `alpha` blend — plus filterable properties and cross-references. Wired together naively, both arms of that hybrid are crippled: figures never become text either arm can match, and fragments lose the heading lineage that carries their defining keywords. The missing link is POMA: `PrimeCut().ingest()` consumes the saved Marker JSON (auto-detected), splices the side-channel images back in, and emits [chunksets](/learn/chunking/chunksets) — self-explanatory root-to-leaf units that give BM25F real keywords, the vector arm real meaning, and Weaviate's property filters real hierarchy fields.

## What each end of the pipeline actually provides

**Marker** (by Datalab / VikParuchuri) runs on your own hardware and produces structured output: with the JSON renderer (`--output_format json`), a Document block tree with Page children, per-block HTML, and full `<table>` elements with row and column spans. What it doesn't provide: cross-page hierarchy, retrieval units, or inline images — figure bytes live in a side-channel `images` dict with only `![](name)` references left in the text. Details: [The Optimal Chunker for Marker](/optimal-chunker-marker).

**Weaviate** provides collections with named vectors, built-in hybrid search that fuses BM25F and vector scores under a tunable `alpha`, filterable properties, cross-references between collections, and generative/reranker modules. What it doesn't provide: any opinion about the objects you store. Hybrid search scores whatever text and vectors you gave it — and both arms operate on *text*, so content that never became text is invisible to both. Details: [The Optimal Chunks for the Best Retrieval in Weaviate](/optimal-chunks-weaviate).

## The naive wiring, and where it breaks

The common recipe — split `rendered.markdown` with a character splitter, embed, insert each fragment as an object — breaks in a way specific to this pair:

- **Both arms of hybrid search go blind on figures.** Weaviate's headline feature is that BM25F catches exact terms the vector arm blurs. But Marker's figure content sits in the side-channel `images` dict; the stored text carries only dangling `![](name)` references. Keyword arm: nothing to match. Vector arm: nothing to embed. The hybrid blend can't rescue content that never became text.
- **BM25F matches land on orphans.** A section's defining keywords often live in its headings — `## Termination clauses`, `### Early termination`. Character-split fragments shed that lineage, so the keyword arm either misses (terms were in a heading the fragment lost) or hits a fragment that arrives context-free in the prompt.
- **No `page` property to filter on.** Marker's markdown output has no page boundaries, so `Filter.by_property("page")` has nothing to work with and answers can't cite pages.
- **Overlap skews both scores.** Duplicated boundary spans produce near-identical objects that crowd top-k under both BM25F and vector scoring.

## The pipeline, end to end

```bash
pip install poma weaviate-client sentence-transformers

# Your existing Marker run — unchanged. The JSON renderer keeps the
# Document block tree that POMA auto-detects, pages included.
marker_single contract.pdf --output_format json --output_dir out/
```

```python
import os
import weaviate
from weaviate.classes.config import Configure, DataType, Property
from weaviate.classes.query import Filter
from poma import PrimeCut
from sentence_transformers import SentenceTransformer

# 1. The missing link — raw Marker JSON in, chunks + chunksets out.
#    Side-channel image bytes are spliced into their ![](name) refs
#    and described into searchable text before chunking.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest("out/contract/contract.json")  # block tree auto-detected

# 2. A chunkset collection with filterable hierarchy properties.
client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=weaviate.auth.AuthApiKey(os.environ["WEAVIATE_API_KEY"]),
)
chunksets = client.collections.create(
    "Chunkset",
    properties=[
        Property(name="content", data_type=DataType.TEXT),
        Property(name="file_id", data_type=DataType.TEXT),
        Property(name="page", data_type=DataType.INT),
        Property(name="depth", data_type=DataType.INT),
        Property(name="chunkset_index", data_type=DataType.INT),
    ],
    vectorizer_config=Configure.Vectorizer.none(),  # we bring our own vectors
)

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
with chunksets.batch.dynamic() as batch:
    for i, cs in enumerate(result.chunksets):
        batch.add_object(
            properties={
                "content": cs.to_embed,
                "file_id": cs.file_id,
                "page": cs.page,
                "depth": cs.depth,
                "chunkset_index": i,
            },
            vector=model.encode(cs.to_embed).tolist(),
        )

# 3. Hybrid retrieval: BM25F + vector, fused by alpha.
query = "What are the early termination conditions?"
hits = chunksets.query.hybrid(
    query=query,                                  # feeds the BM25F arm
    vector=model.encode(query).tolist(),          # feeds the vector arm
    alpha=0.5,                                    # balanced blend; tune per query mix
    limit=5,
    filters=Filter.by_property("file_id").equal(result.chunksets[0].file_id),
)
retrieved = [o.properties["chunkset_index"] for o in hits.objects]
# Look up those chunksets in the .poma archive and merge them into one
# deduplicated, prompt-ready cheatsheet — hierarchy included.
```

POMA validates the payload up front — the JSON Document block tree is the strict auto-route fingerprint; a corrupted or mislabeled upload 422s immediately. Bare markdown shapes (`{markdown, images, metadata}`, the deprecated `{output, format}` envelope) need an explicit `external_ocr_source="marker"`. Because every chunkset carries its full root-to-leaf lineage in `content`, the BM25F arm can match heading terms that character splitters strand in other fragments — the keyword arm gets structurally better input, not just cleaner text. And overlap-free chunksets plus cheatsheet assembly keep retrieved context lean: on our reference legal document, **337 tokens** versus **1,542** for a recursive character splitter, with zero information loss — methodology in [Document Ingestion & Chunking for RAG](/document-ingestion-chunking-rag).

## Metadata mapping: POMA fields → Weaviate primitives

| POMA chunk field | Weaviate primitive | What it enables |
| --- | --- | --- |
| `to_embed` | `content` TEXT property + object vector | BM25F keyword arm + vector arm of hybrid |
| `file_id` | TEXT property, `Filter.by_property("file_id")` | scope queries to one document |
| `page` (from Marker's Page blocks, JSON renderer) | INT property, range filters | page-cited answers, page-range scoping |
| `depth` | INT property | filter/re-rank by hierarchy level |
| `chunk_index` / chunkset index | INT property | ID-based lookup in the `.poma` archive |
| chunk ↔ chunkset lineage | optional **cross-reference** between collections | traversable hierarchy inside Weaviate |

## Frequently asked questions

### How do I get Marker output into Weaviate for RAG?

Run `marker_single --output_format json`, hand the saved result to `PrimeCut().ingest()` (auto-detected, images spliced, hierarchy rebuilt), create a collection with `content`/`file_id`/`page`/`depth` properties, insert each chunkset with its vector, and query with `collection.query.hybrid` so both arms work together.

### Do Marker figures show up in Weaviate hybrid search?

Not by themselves — both hybrid arms operate on text, and Marker's figure bytes sit in a side-channel `images` dict behind bare `![](name)` refs. POMA splices the bytes in as data URIs and describes each figure into searchable text, so charts become findable by keyword and by meaning. Refs without bytes are neutralized and counted.

### What alpha should I use for hybrid search over Marker-parsed documents?

Start balanced at `alpha=0.5` and tune. Locally parsed contracts and manuals carry exact tokens (clause numbers, part codes, defined terms) that make the BM25F arm genuinely valuable — provided the stored text is self-explanatory chunksets that still carry their heading keywords, not orphaned fragments.

### Which Weaviate properties should Marker chunks map to?

`content` (the `to_embed` text), `file_id` (TEXT, filterable), `page` (INT, from the JSON renderer's Page blocks), `depth` (INT), and `chunkset_index` (INT) for cheatsheet assembly. Filter with `Filter.by_property` at query time.

### Can I model the chunk-to-chunkset relation with Weaviate cross-references?

Yes — a Chunk collection cross-referenced from a Chunkset collection makes lineage traversable inside Weaviate. It's an optimization, though: chunksets are already self-explanatory, and cheatsheet assembly from the `.poma` archive by ID covers the common case.

## Related recipes

Same parser, different store: [Marker → Qdrant](/pipelines/marker-to-qdrant) · [Marker → Pinecone](/pipelines/marker-to-pinecone) · [Marker → Chroma](/pipelines/marker-to-chroma)

Same store, different parser: [Mistral OCR → Weaviate](/pipelines/mistral-ocr-to-weaviate) · [Unstructured → Weaviate](/pipelines/unstructured-to-weaviate) · [Docling → Weaviate](/pipelines/docling-to-weaviate)

Foundations: [The Optimal Chunker for Marker](/optimal-chunker-marker) · [The Optimal Chunks for Weaviate](/optimal-chunks-weaviate) · [All pipeline recipes](/pipelines/) · [RAG chunking guide](/guides/rag-chunking/)