Grill Ingestion

Grill's ingestion entry point is POST /grill/ingest. It runs the same PrimeCut pipeline — parse, chunk, embed — but persists the result inside your project's namespace instead of returning a .poma archive. Once a job finishes the document is immediately searchable through /grill/search; you never download or re-upload anything.

Why a separate endpoint

POST /grill/ingest and POST /primeCut/ingest look almost identical on the wire. They differ in what happens after the chunks are produced:

Step	/primeCut/ingest	/grill/ingest
Parse + chunk	✅ same pipeline	✅ same pipeline
Embed	Optional, depends on plan	✅ always
Persist to project namespace (vectors + storage)	❌	✅
Make doc available to /grill/search	❌	✅
.poma archive download via /jobs/{job_id}/download	✅	❌

If you want chunks to take home, use PrimeCut. If you want the document searchable through /grill/search, use Grill.

One project = one product. A project created with product:"primecut" cannot call /grill/ingest, and vice versa. See Create a Grill project.

The job lifecycle

Ingestion is asynchronous. The job_id you get back moves through the standard POMA lifecycle:

pending  ──▶ processing  ──▶ done       ◀── searchable from this point
                          └─▶ failed

At done, three things are true atomically: the document appears in GET /grill/docs, GET /grill/docs/{docId} returns its DocInfo, and /grill/search can retrieve from it. The SDK's ingest() waits for this for you; with raw HTTP you poll GET /jobs/{job_id}/status or stream GET /status/v1/jobs/{job_id}. There's no .poma to download — Grill keeps the artifacts server-side.

docId is derived from the filename (sanitised) plus a project salt; read the canonical value from DocInfo.doc_id and use it for doc_filter and /grill/docs/{docId}.

Ingest with the Python SDK

The recommended path. The Grill client handles the octet-stream framing, polling, and status streaming for you, and reads the project API key from POMA_GRILL_API_KEY.

# pip install poma
from poma import Grill

g = Grill()
result = g.ingest("manual.pdf")        # submit + wait, returns when done
print(result.job_id, result.status, result.usage)

Batch — submit everything first, collect later so the waits overlap:

g = Grill()
job_ids = [g.submit(p) for p in ["a.pdf", "b.pdf", "c.pdf"]]
results = [g.collect(jid) for jid in job_ids]

Async — run the waits concurrently with AsyncGrill:

import asyncio
from poma import AsyncGrill

async def ingest_all(paths: list[str]) -> None:
    async with AsyncGrill() as g:
        results = await asyncio.gather(*(g.ingest(p) for p in paths))
        for r in results:
            print(r.job_id, r.status)

asyncio.run(ingest_all(["a.pdf", "b.pdf", "c.pdf"]))

From a URL instead of a local file — pass remote_url and the server fetches it:

g.ingest(remote_url="https://example.com/report.pdf")

Full signatures: Grill reference, AsyncGrill reference.

Labels & metadata for filtering

Tag a document at ingest so queries can filter on it later. With the SDK these are keyword arguments on ingest() / submit():

g.ingest(
    "annual-report-2025.pdf",
    labels=["year:2025", "source:investor-relations"],   # categorical tags
    meta_int_1=1735689600,                                # e.g. a Unix timestamp
    meta_int_2=3,                                          # e.g. a revision number
)

Each maps to an ingest header and a search-time filter:

SDK argument	Ingest header	Query-time filter	Use for
labels=[…]	X-Labels	labels_any / labels_all (SDK) · meta_tags_any / meta_tags_all (HTTP)	Categorical tags — "year:2025", "source:treasury". Max 64, ≤ 128 chars each. Stored HMAC'd / opaque, so matching is equality-only.
meta_int_1= / meta_int_2=	X-Meta-Int-1 / X-Meta-Int-2	meta_int_1_gte / _lte, meta_int_2_gte / _lte	Range-queryable integers — a timestamp, a revision number. Stored plaintext.

Pick labels by sensitivity: X-Labels values are HMAC'd before they reach the vector DB, so the plaintext is never stored — safe for sensitive tags, at the cost of equality-only matching. The HTTP API also accepts X-Unencrypted-Strings for plaintext, wildcard-filterable metadata (see the API reference) — never put secrets or PII there.

See Retrieval for how to apply these filters at search time.

Direct API usage

If you're not on Python, POST /grill/ingest takes raw file bytes as application/octet-stream, with the filename in Content-Disposition:

curl -sS -X POST "$GRILL/ingest" \
  -H "authorization: Bearer $GRILL_KEY" \
  -H "content-type: application/octet-stream" \
  -H 'content-disposition: attachment; filename="manual.pdf"' \
  --data-binary @manual.pdf

• Multipart (multipart/form-data) is not accepted → 403. Use octet-stream.
• Fetch from a URL instead of uploading bytes by setting X-Remote-URL (then Content-Disposition and the body are optional).
• Labels & metadata ride as headers: X-Labels, X-Meta-Int-1 / X-Meta-Int-2, X-Unencrypted-Strings, plus X-Base-URL and X-Completion (completion webhook).

Full request/response shapes and every header are in the API reference.

Supported file types

Grill inherits the full PrimeCut format set:

• Documents: pdf, doc, docx, dotx, rtf, txt, md, html, htm, xml
• Data & structured text: json, yaml, toml, ini, env, cir
• Presentations: ppt, pptx, pptm, pps, ppsx, pot, potx, key
• Spreadsheets: xls, xlsx, xlsm, xlsb, xltx, csv, tsv, numbers, ods, odc
• Images: png, jpg, jpeg, gif, bmp, tif, tiff, svg, webp, ico, heic, heif, psd
• Audio: mp3, wav, m4a, aac, ogg, flac, opus
• Video: mp4, mov, webm, mkv, avi, m4v, mpeg, mpg
• Other: epub, mobi, djvu, dwg, dxf, dwf, dwfx, vsd, vsdx, ai, eps, ps, prn, xps, oxps, pub, mdi, pages, odp, odf, odt

Audio & video are handled by a native media front-end — video is keyframe-sampled and transcribed (speech-to-text) with per-keyframe vision, audio runs the same understanding pass without frames — so both become searchable text like any other document.

Ingesting from a URL: set X-Remote-URL to have Grill fetch the bytes instead of uploading them (the Content-Disposition and body become optional). Public video-platform URLs (e.g. YouTube) are supported as a source too.

Re-ingesting the same file

Re-uploading a file with the same effective docId replaces the existing document — old vectors and storage are discarded. There's no append mode; an updated PDF fully supersedes the previous version. For version history, ingest each version under a distinct filename so the docId differs. To remove a doc cleanly first, use DELETE /grill/docs/{docId} (Document management).

Errors you will see

Status	When	What to do
400	No X-Remote-URL and a missing/invalid Content-Disposition, unsupported MIME, or empty body	Fix the headers; check the file is non-empty (or supply X-Remote-URL).
401	Missing or invalid Bearer token	Use a project API key — see Authentication.
403	Caller's project is primecut, or the request is multipart	Create a Grill project; switch to octet-stream.
500	Server-side parse failure	Retry once; if it persists, contact support with the job_id.

Next

• Retrieval — once the doc is in, how search behaves.
• Document management — list, inspect, delete.
• API reference — full /grill/ingest request/response.