Skip to content

LlamaParse Chunking for RAG: From Raw Result to Chunksets ​

The short answer: the optimal chunker for LlamaParse is one that consumes the saved result JSON as-is, works from the md field (not the flattened text), rebuilds the heading hierarchy across pages, and emits retrieval units that carry their own context. POMA's bring-your-own-OCR connector does exactly this: you upload the unmodified LlamaParse JSON result, and PrimeCut returns hierarchy-preserving chunks and chunksets — no re-parsing, no markdown flattening, no MarkdownElementNodeParser-plus-fixed-splitter workaround.

This page explains what LlamaParse gives you, where it stops, why the default LlamaIndex chunking step squanders it, and how to go from a LlamaParse job to embedded chunksets in a few lines.

What LlamaParse gives you — and where it stops ​

LlamaParse, LlamaIndex's document parser, is one of the best PDF-to-markdown parsers around — it's markdown-native by design, and its output reflects that. A parse job returns per-page results in a JSON envelope:

json
{
  "pages": [
    {
      "page": 1,
      "md": "# Annual Report 2025\n\n## Financial highlights\n\n| Metric | FY25 |\n| --- | --- |\n…",
      "text": "Annual Report 2025\nFinancial highlights\n…",
      "images": [{ "name": "img_p1_1.png", "height": 480, "width": 640 }]
    }
  ]
}

Each page carries two renderings of the same content: md (markdown — headings marked, tables inline) and text (a plain flattening). The md field is where the value lives, and it's why LlamaParse's "markdown mode" earned its reputation: headings, lists, and tables are genuinely recovered, page by page.

What LlamaParse deliberately does not do:

  • Cross-page hierarchy. Each page's md is independent. A ## Termination clauses heading on page 41 has no machine-readable link to the # Master Services Agreement chapter that opened on page 3.
  • Chunking. The result is display-ready markdown, not retrieval-ready units. LlamaIndex hands you node parsers for the next step — and that step is where most pipelines go wrong.
  • Inline image bytes. Image content stays server-side at LlamaParse; each image needs a separate /result/image/{name} fetch. A saved result JSON contains only ![](name) references — something downstream has to deal with those dead links, or at least make their loss visible.

LlamaParse's job ends at "here is markdown, per page." The gap between markdown and good retrieval is where RAG pipelines quietly fail.

Why the default LlamaIndex chunking step wastes LlamaParse output ​

The canonical pattern — parse with LlamaParse, then feed the markdown to MarkdownElementNodeParser, or worse, concatenate pages[].md and run a fixed-size SentenceSplitter — destroys most of what you just paid for:

What LlamaParse recoveredWhat the default node parser / splitter does with it
Heading levels (#, ##, ###)Used as cut points at best — ancestor context discarded after the cut
Inline markdown tablesMarkdownElementNodeParser isolates them; fixed splitters slice them mid-row
Page boundaries (pages[].page)Lost at concatenation — nodes can't cite a page
The md vs text distinctionPipelines that grab text throw away all structure before chunking even starts
Image referencesDead ![](name) links either pollute nodes or get regex-stripped silently

MarkdownElementNodeParser is the honest baseline here — it does separate tables from prose and respects heading boundaries within its window. But the nodes it emits are isolated fragments: the node under ### Early termination no longer knows it lives inside ## Termination clauses inside # Master Services Agreement. At retrieval time the LLM reads an orphaned paragraph and answers out of context. That failure mode — not parsing accuracy — is where most LlamaParse-based RAG pipelines lose their quality.

For the full taxonomy of these failure modes, see the RAG chunking strategies guide.

The optimal chunker: hierarchy-aware chunking on the raw result ​

POMA's bring-your-own-OCR (BYOCR) connector was built for teams who already run their own parsing step. The connector consumes the unmodified saved LlamaParse JSON result and runs only POMA's downstream stages on it:

  1. Shape validation, up front. The payload is fingerprinted (pages[] with md + page keys). A corrupted or mislabeled upload fails immediately with a clear 422 — never a silently degraded index.
  2. md over text, always. POMA reads the markdown rendering, where heading levels are marked and tables arrive inline — the plain text flattening is ignored, so no structure is lost before chunking begins.
  3. Offloaded images made visible. Because LlamaParse keeps image bytes server-side, the saved JSON carries only ![](name) references. POMA neutralizes these dead refs so they don't pollute chunks, and counts every one in content_metadata — content loss is surfaced and quantified, never silent.
  4. Noise removal. Running headers and footers that repeat on every page are stripped — the same pass POMA's native pipeline applies — so your index isn't 4% page furniture.
  5. Heading enrichment. Where the parsed markdown under-marks headings (common in scanned or design-heavy documents), POMA's heading-injection pass restores them before chunking — the same treatment every natively ingested document gets.
  6. Hierarchy → chunks + chunksets. The cross-page heading tree is rebuilt, and every sentence is emitted as a chunk with its depth, page, and a to_embed normalization — grouped into chunksets: root-to-leaf paths that keep each retrieved passage self-explanatory.

Because you already ran the parsing, POMA charges only for the downstream structure and chunking value — the OCR front-end is skipped entirely.

The result, on one reference legal document: the same query answered with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.

How to chunk LlamaParse output with POMA ​

Run your parse job exactly as you do today, save the raw JSON result, and hand it to PrimeCut:

python
import json
import os
from llama_parse import LlamaParse
from poma import PrimeCut

# 1. Your existing LlamaParse call — unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
    json.dump(json_result[0], f)

# 2. Chunk the raw result — POMA auto-detects the LlamaParse shape.
client = PrimeCut()  # reads POMA_API_KEY
result = client.ingest("contract.llamaparse.json")

print(f"chunks: {len(result.chunks)}")
print(f"chunksets: {len(result.chunksets)}")
print(result.chunksets[0].to_embed)  # embedding-ready, hierarchy included

Two things worth knowing:

  • Auto-detection. A JSON upload whose shape structurally fingerprints as a LlamaParse result (pages[] carrying md + page) is routed through the BYOCR seam automatically. To force the declaration — or to opt a look-alike JSON out of detection — set the external_ocr_source parameter to "llamaparse" or "none" respectively on the ingest request.
  • Dedicated endpoint. For explicit routing, the API exposes a chunk_external_ocr_result endpoint that accepts external_ocr_source: "llamaparse". See the API reference.

If you're building on LlamaIndex, nothing else changes: POMA replaces the node-parser step, not your framework. From result.chunks / result.chunksets onward, everything is your stack — see the LlamaIndex, LangChain, and Qdrant integrations.

Chunking options for LlamaParse output, compared ​

ApproachStructure usedCross-page hierarchyTablesImagesPage numbers kept
Fixed-size splitter (SentenceSplitter et al.)None✗Sliced mid-rowDead refs inlined as noise✗
MarkdownElementNodeParserPer-page headings + table elements✗Isolated as elementsUsually strippedManual
Semantic chunkerEmbedding similarity✗FragileStrippedManual
POMA BYOCR (PrimeCut)Full heading tree, rebuilt across pages✓ chunksetsInline markdown preserved, never cutNeutralized and visibly counted✓ per chunk

Frequently asked questions ​

How do I chunk LlamaParse output for RAG? ​

Don't concatenate the per-page markdown and run a fixed-size splitter — you'd discard the page boundaries and heading structure LlamaParse already recovered. Feed the raw result JSON to a structure-aware chunker instead. POMA's BYOCR connector accepts the saved JSON unmodified, rebuilds the heading hierarchy across pages, and emits chunks plus chunksets ready for embedding.

Can I use LlamaParse results with POMA without re-parsing? ​

Yes — that's the point of the bring-your-own-OCR path. POMA skips its OCR front-end and runs only the downstream structure, chunking, and retrieval-preparation stages on your result. Upload the saved JSON (auto-detected) or declare external_ocr_source: "llamaparse".

What does LlamaParse return, exactly? ​

A pages array where each page carries a page number, an md string (markdown, tables inline), a text string (plain flattening), and an images list. Image bytes stay server-side at LlamaParse; the saved JSON has only references. Structure is per-page; nothing links headings across pages — that's the chunker's job.

Should I chunk the md or the text field from a LlamaParse result? ​

Use md. It carries the heading levels and inline tables that make hierarchy-aware chunking possible; text is a plain flattening that discards both. POMA's connector prefers md over text automatically.

What happens to images in a LlamaParse result during chunking? ​

The saved result JSON contains only ![](name) references — the bytes live server-side and require a separate /result/image/{name} fetch per image. POMA neutralizes those dead refs and counts each one in content_metadata. Image loss is always visible and quantified, never silent.

Is MarkdownElementNodeParser enough for chunking LlamaParse output? ​

It separates tables from prose, but its nodes are isolated fragments with no ancestor context — a subsection node doesn't know which chapter it belongs to. A hierarchy-preserving chunker keeps every passage attached to its full heading path, so retrieved context stays self-explanatory.

The rest of the series ​

The optimal chunker, for every OCR and parsing tool you already run:

Wire LlamaParse straight into your vector database:

Or start with the fundamentals: RAG chunking guide · Document ingestion guide · POMA chunksets.