LlamaParse Chunking for RAG: From Raw Result to Chunksets
The short answer: the optimal chunker for LlamaParse is one that consumes the saved result JSON as-is, works from the md field (not the flattened text), rebuilds the heading hierarchy across pages, and emits retrieval units that carry their own context. POMA's bring-your-own-OCR connector does exactly this: you upload the unmodified LlamaParse JSON result, and PrimeCut returns hierarchy-preserving chunks and chunksets — no re-parsing, no markdown flattening, no MarkdownElementNodeParser-plus-fixed-splitter workaround.
This page explains what LlamaParse gives you, where it stops, why the default LlamaIndex chunking step squanders it, and how to go from a LlamaParse job to embedded chunksets in a few lines.
What LlamaParse gives you — and where it stops
LlamaParse, LlamaIndex's document parser, is one of the best PDF-to-markdown parsers around — it's markdown-native by design, and its output reflects that. A parse job returns per-page results in a JSON envelope:
{
"pages": [
{
"page": 1,
"md": "# Annual Report 2025\n\n## Financial highlights\n\n| Metric | FY25 |\n| --- | --- |\n…",
"text": "Annual Report 2025\nFinancial highlights\n…",
"images": [{ "name": "img_p1_1.png", "height": 480, "width": 640 }]
}
]
}Each page carries two renderings of the same content: md (markdown — headings marked, tables inline) and text (a plain flattening). The md field is where the value lives, and it's why LlamaParse's "markdown mode" earned its reputation: headings, lists, and tables are genuinely recovered, page by page.
What LlamaParse deliberately does not do:
- Cross-page hierarchy. Each page's
mdis independent. A## Termination clausesheading on page 41 has no machine-readable link to the# Master Services Agreementchapter that opened on page 3. - Chunking. The result is display-ready markdown, not retrieval-ready units. LlamaIndex hands you node parsers for the next step — and that step is where most pipelines go wrong.
- Inline image bytes. Image content stays server-side at LlamaParse; each image needs a separate
/result/image/{name}fetch. A saved result JSON contains onlyreferences — something downstream has to deal with those dead links, or at least make their loss visible.
LlamaParse's job ends at "here is markdown, per page." The gap between markdown and good retrieval is where RAG pipelines quietly fail.
Why the default LlamaIndex chunking step wastes LlamaParse output
The canonical pattern — parse with LlamaParse, then feed the markdown to MarkdownElementNodeParser, or worse, concatenate pages[].md and run a fixed-size SentenceSplitter — destroys most of what you just paid for:
| What LlamaParse recovered | What the default node parser / splitter does with it |
|---|---|
Heading levels (#, ##, ###) | Used as cut points at best — ancestor context discarded after the cut |
| Inline markdown tables | MarkdownElementNodeParser isolates them; fixed splitters slice them mid-row |
Page boundaries (pages[].page) | Lost at concatenation — nodes can't cite a page |
The md vs text distinction | Pipelines that grab text throw away all structure before chunking even starts |
| Image references | Dead  links either pollute nodes or get regex-stripped silently |
MarkdownElementNodeParser is the honest baseline here — it does separate tables from prose and respects heading boundaries within its window. But the nodes it emits are isolated fragments: the node under ### Early termination no longer knows it lives inside ## Termination clauses inside # Master Services Agreement. At retrieval time the LLM reads an orphaned paragraph and answers out of context. That failure mode — not parsing accuracy — is where most LlamaParse-based RAG pipelines lose their quality.
For the full taxonomy of these failure modes, see the RAG chunking strategies guide.
The optimal chunker: hierarchy-aware chunking on the raw result
POMA's bring-your-own-OCR (BYOCR) connector was built for teams who already run their own parsing step. The connector consumes the unmodified saved LlamaParse JSON result and runs only POMA's downstream stages on it:
- Shape validation, up front. The payload is fingerprinted (
pages[]withmd+pagekeys). A corrupted or mislabeled upload fails immediately with a clear 422 — never a silently degraded index. mdovertext, always. POMA reads the markdown rendering, where heading levels are marked and tables arrive inline — the plaintextflattening is ignored, so no structure is lost before chunking begins.- Offloaded images made visible. Because LlamaParse keeps image bytes server-side, the saved JSON carries only
references. POMA neutralizes these dead refs so they don't pollute chunks, and counts every one incontent_metadata— content loss is surfaced and quantified, never silent. - Noise removal. Running headers and footers that repeat on every page are stripped — the same pass POMA's native pipeline applies — so your index isn't 4% page furniture.
- Heading enrichment. Where the parsed markdown under-marks headings (common in scanned or design-heavy documents), POMA's heading-injection pass restores them before chunking — the same treatment every natively ingested document gets.
- Hierarchy → chunks + chunksets. The cross-page heading tree is rebuilt, and every sentence is emitted as a chunk with its
depth,page, and ato_embednormalization — grouped into chunksets: root-to-leaf paths that keep each retrieved passage self-explanatory.
Because you already ran the parsing, POMA charges only for the downstream structure and chunking value — the OCR front-end is skipped entirely.
The result, on one reference legal document: the same query answered with 337 tokens of retrieved context instead of 1,542 for a recursive character splitter, with nothing lost. One document is an illustration, not a benchmark; the ingestion guide has the broader numbers.
How to chunk LlamaParse output with POMA
Run your parse job exactly as you do today, save the raw JSON result, and hand it to PrimeCut:
import json
import os
from llama_parse import LlamaParse
from poma import PrimeCut
# 1. Your existing LlamaParse call — unchanged.
parser = LlamaParse(api_key=os.environ["LLAMA_CLOUD_API_KEY"])
json_result = parser.get_json_result("contract.pdf")
with open("contract.llamaparse.json", "w") as f:
json.dump(json_result[0], f)
# 2. Chunk the raw result — POMA auto-detects the LlamaParse shape.
client = PrimeCut() # reads POMA_API_KEY
result = client.ingest("contract.llamaparse.json")
print(f"chunks: {len(result.chunks)}")
print(f"chunksets: {len(result.chunksets)}")
print(result.chunksets[0].to_embed) # embedding-ready, hierarchy includedTwo things worth knowing:
- Auto-detection. A JSON upload whose shape structurally fingerprints as a LlamaParse result (
pages[]carryingmd+page) is routed through the BYOCR seam automatically. To force the declaration — or to opt a look-alike JSON out of detection — set theexternal_ocr_sourceparameter to"llamaparse"or"none"respectively on the ingest request. - Dedicated endpoint. For explicit routing, the API exposes a
chunk_external_ocr_resultendpoint that acceptsexternal_ocr_source: "llamaparse". See the API reference.
If you're building on LlamaIndex, nothing else changes: POMA replaces the node-parser step, not your framework. From result.chunks / result.chunksets onward, everything is your stack — see the LlamaIndex, LangChain, and Qdrant integrations.
Chunking options for LlamaParse output, compared
| Approach | Structure used | Cross-page hierarchy | Tables | Images | Page numbers kept |
|---|---|---|---|---|---|
| Fixed-size splitter (SentenceSplitter et al.) | None | ✗ | Sliced mid-row | Dead refs inlined as noise | ✗ |
| MarkdownElementNodeParser | Per-page headings + table elements | ✗ | Isolated as elements | Usually stripped | Manual |
| Semantic chunker | Embedding similarity | ✗ | Fragile | Stripped | Manual |
| POMA BYOCR (PrimeCut) | Full heading tree, rebuilt across pages | ✓ chunksets | Inline markdown preserved, never cut | Neutralized and visibly counted | ✓ per chunk |
Frequently asked questions
How do I chunk LlamaParse output for RAG?
Don't concatenate the per-page markdown and run a fixed-size splitter — you'd discard the page boundaries and heading structure LlamaParse already recovered. Feed the raw result JSON to a structure-aware chunker instead. POMA's BYOCR connector accepts the saved JSON unmodified, rebuilds the heading hierarchy across pages, and emits chunks plus chunksets ready for embedding.
Can I use LlamaParse results with POMA without re-parsing?
Yes — that's the point of the bring-your-own-OCR path. POMA skips its OCR front-end and runs only the downstream structure, chunking, and retrieval-preparation stages on your result. Upload the saved JSON (auto-detected) or declare external_ocr_source: "llamaparse".
What does LlamaParse return, exactly?
A pages array where each page carries a page number, an md string (markdown, tables inline), a text string (plain flattening), and an images list. Image bytes stay server-side at LlamaParse; the saved JSON has only references. Structure is per-page; nothing links headings across pages — that's the chunker's job.
Should I chunk the md or the text field from a LlamaParse result?
Use md. It carries the heading levels and inline tables that make hierarchy-aware chunking possible; text is a plain flattening that discards both. POMA's connector prefers md over text automatically.
What happens to images in a LlamaParse result during chunking?
The saved result JSON contains only  references — the bytes live server-side and require a separate /result/image/{name} fetch per image. POMA neutralizes those dead refs and counts each one in content_metadata. Image loss is always visible and quantified, never silent.
Is MarkdownElementNodeParser enough for chunking LlamaParse output?
It separates tables from prose, but its nodes are isolated fragments with no ancestor context — a subsection node doesn't know which chapter it belongs to. A hierarchy-preserving chunker keeps every passage attached to its full heading path, so retrieved context stays self-explanatory.
The rest of the series
The optimal chunker, for every OCR and parsing tool you already run:
- AWS Textract
- Azure Document Intelligence
- DeepSeek-OCR
- Docling
- Marker
- MinerU
- Mistral OCR
- PaddleOCR-VL
- Unstructured.io
Wire LlamaParse straight into your vector database:
- LlamaParse → Chroma
- LlamaParse → Elasticsearch
- LlamaParse → LanceDB
- LlamaParse → Milvus
- LlamaParse → MongoDB Atlas
- LlamaParse → OpenSearch
- LlamaParse → pgvector
- LlamaParse → Pinecone
- LlamaParse → Qdrant
- LlamaParse → Redis
- LlamaParse → Turbopuffer
- LlamaParse → Vespa
- LlamaParse → Weaviate
Or start with the fundamentals: RAG chunking guide · Document ingestion guide · POMA chunksets.