Skip to content

LlamaParse to Pinecone: Chunking for RAG ​

The short answer: LlamaParse gives you excellent per-page markdown; Pinecone gives you serverless vector search with namespaces and metadata filters. Wired together the standard LlamaIndex way, they still produce mediocre RAG — isolated nodes, lost page numbers, and node text stuffed into metadata that collides with Pinecone's ~40 KB cap. The missing link is POMA: PrimeCut().ingest() consumes the saved LlamaParse result JSON (auto-detected), rebuilds the cross-page heading hierarchy into chunksets, and you upsert each chunkset's to_embed with compact hierarchy metadata — full text stays in your doc store or the .poma archive, retrieved IDs rebuild into cheatsheets.

What each end of the pipeline actually provides ​

LlamaParse, LlamaIndex's markdown-native parser, returns a JSON result with pages[], each carrying a page number, an md string (headings marked, tables inline), a flattened text string, and an images list — the bytes stay server-side, one /result/image/{name} fetch each. What it doesn't provide: cross-page hierarchy (each page's md is independent) or retrieval units. Details: The Optimal Chunker for LlamaParse.

Pinecone provides serverless indexes, namespaces for physical per-tenant partitioning, metadata filtering ($eq, $in, $gte), sparse-dense hybrid records, and integrated inference — with a metadata budget of roughly 40 KB per vector. What it doesn't provide: any opinion about what a record should contain. It retrieves nearest neighbors of whatever you embedded. Details: The Optimal Chunks for the Best Retrieval in Pinecone.

The naive wiring, and where it breaks ​

The canonical recipe — LlamaParse(result_type="markdown").load_data(...) → MarkdownElementNodeParser → a vector store index over Pinecone — breaks in a pair-specific way:

  • Node text lands in metadata, and the cap bites your best pages first. LlamaIndex vector-store integrations serialize node content and relationships into record metadata. LlamaParse's md pages with large inline tables — the pages you parsed a financial report for — can run to tens of kilobytes, and those are exactly the upserts that fail against the ~40 KB metadata limit. Depending on your error handling, the result is a hard 4xx or a corpus that silently lacks its densest pages.
  • Nodes arrive as orphans. MarkdownElementNodeParser separates tables from prose, but its nodes carry no ancestor context — the fragment under ### Early termination no longer knows which agreement it terminates.
  • LlamaParse's page numbers never reach the filter. By the time nodes exist, pages[].page is gone, so filter={"page": ...} has nothing to match and answers can't cite pages.
  • Dead image refs pollute the embedding. The saved result carries only ![](name) references; fed to a node parser, they become noise in the embedded text or vanish without a trace.

Neither tool is at fault. LlamaParse parsed correctly; Pinecone enforced a documented limit. The step between them put the wrong thing in the wrong place.

The pipeline, end to end ​

bash
pip install llama-parse poma pinecone
python
import json
import os

from llama_parse import LlamaParse
from pinecone import Pinecone
from poma import PrimeCut
from poma.utils import unpack_poma_archive

# 1. LlamaParse — your existing parse job, unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
    json.dump(json_result[0], f)

# 2. The missing link — raw result JSON in, chunks + chunksets out.
poma = PrimeCut()  # reads POMA_API_KEY
result = poma.ingest(
    "contract.llamaparse.json",  # LlamaParse shape auto-detected
    download_dir="store",
    filename="contract.poma",    # keep the archive — it carries the page numbers
)

# Page numbers live on the archive's chunk records; the SDK's typed objects
# expose depth but not page. Look both up by chunk index.
archive = unpack_poma_archive(poma_archive_path="store/contract.poma")
page_of = {c["chunk_index"]: c.get("page") for c in archive["chunks"]}
depth_of = {c.chunk_index: c.depth for c in result.chunks}

# 3. Pinecone — compact metadata; the full text stays in your doc store.
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
index = pc.Index("contracts")

records = []
for cs in result.chunksets:
    leaf = cs.chunks[-1]  # chunk indices in document order; the last one is the leaf
    records.append({
        "id": f"{cs.file_id}:{cs.chunkset_index}",
        "values": embed(cs.to_embed),  # your embedding model
        "metadata": {
            "file_id": cs.file_id,
            "page": page_of[leaf],  # LlamaParse pages[].page, via the archive's chunks
            "depth": depth_of[leaf],
            "chunkset_index": cs.chunkset_index,
        },
    })
index.upsert(vectors=records, namespace="acme-corp")

# 4. Query with a filter, then rebuild cheatsheets from the returned IDs.
matches = index.query(
    vector=embed("What are the early termination conditions?"),
    top_k=5,
    namespace="acme-corp",
    filter={"file_id": {"$eq": result.chunksets[0].file_id}},
    include_metadata=True,
)
hit_ids = [m["id"] for m in matches["matches"]]
# Look up the full chunksets by ID in the .poma archive or your doc store,
# deduplicate the shared heading lineage, and assemble one cheatsheet.

POMA fingerprints the payload up front (pages[] with md + page — a corrupted or mislabeled upload 422s immediately), reads md and ignores the flattened text, neutralizes the server-side image refs and counts them in content_metadata, strips running headers and footers, and rebuilds the cross-page heading tree before chunking. To force or suppress detection, set external_ocr_source to "llamaparse" or "none". If you use Pinecone's integrated inference, send cs.to_embed as the text field and let the hosted model embed it — the unit matters more than where the embedding runs. For hybrid retrieval, compute a sparse term-weight vector from the same to_embed string so both signals describe the identical unit.

The record ID {file_id}:{chunkset_index} is the join key back to the chunkset in the .poma archive or your document store. That lookup-then-merge step — deduplicating the ancestors retrieved chunksets share — is what POMA calls a cheatsheet, and it is where the token savings land: 337 tokens of context versus 1,542 for a recursive-splitter baseline on our reference legal document, with zero information loss — methodology here.

Metadata mapping: POMA fields → Pinecone primitives ​

POMA chunk fieldPinecone primitiveWhat it enables
to_embeddense vector values (+ optional sparse values from the same string)paraphrase + exact-term hybrid retrieval
file_idmetadata field, filterable ($eq, $in)scope queries to one document
page (on the archive's chunk records, derived from LlamaParse pages[].page; take the chunkset's leaf chunk)metadata field, filterable ($eq, $gte)page-cited answers, page-range filters
depth (of the chunkset's leaf chunk)metadata field, filterableexclude deep appendix content, re-rank by level
chunkset_indexmetadata field + half of the record IDstable ordering, join key to the .poma archive
chunkset full textnot metadata — doc store / .poma, keyed by IDcheatsheet assembly without brushing the ~40 KB cap

Frequently asked questions ​

How do I get LlamaParse output into Pinecone for RAG? ​

Save the raw JSON result, run PrimeCut().ingest() on it (auto-detected, md preferred, hierarchy rebuilt), then embed each chunkset's to_embed and upsert with compact file_id/page/depth/chunkset_index metadata — page comes from the archive's chunk records (the chunkset's leaf chunk), depth from the matching typed chunk. Full text stays in your doc store or the .poma archive, keyed by vector ID.

Why do my LlamaParse upserts hit Pinecone's metadata limit? ​

Because node text is being serialized into metadata, and LlamaParse md pages with big inline tables can run to tens of kilobytes — your most information-dense pages fail the ~40 KB cap first. Store compact fields in metadata and keep the text outside, keyed by ID.

What metadata should LlamaParse chunks carry in Pinecone? ​

file_id, page (from pages[].page, read off the archive's chunk records — take the chunkset's leaf chunk), depth, chunkset_index — compact fields that power $eq/$in/$gte filters and stay far under the cap. POMA records the page number on every chunk in the .poma archive; concatenate-then-split pipelines lose it.

How should I use Pinecone namespaces for LlamaParse documents? ​

One namespace per tenant or corpus; metadata filters select the document inside it. The namespace answers "whose data", the file_id filter answers "which document" — don't emulate one mechanism with the other.

Do images in a LlamaParse result survive the trip to Pinecone? ​

Not as content — the bytes live server-side at LlamaParse and the saved JSON has only ![](name) refs. POMA neutralizes those dead refs and counts each in content_metadata, so the loss is visible and quantified, never silently embedded.

Same parser, different store: LlamaParse → Qdrant · LlamaParse → Weaviate · LlamaParse → Milvus

Same store, different parser: Mistral OCR → Pinecone · Unstructured.io → Pinecone · Textract → Pinecone

Foundations: The Optimal Chunker for LlamaParse · The Optimal Chunks for Pinecone · All pipeline recipes · RAG chunking guide