Skip to content
Sign inStart chatting
Menu

Run large jobs for less.

Send many requests in one file. They run within 24 hours on spare GPU time, and cost less per token.

Start a batch Use the API

How batches work.

  • One file

    One request per line, in the OpenAI batch format your SDK already writes. Up to 10,000 requests.

  • Within 24 hours

    Requests run on spare GPU time, so they cost less than the same requests sent live.

  • Answers as one file

    Download the answers and a file of any errors, from the console or the API.

Batches cost less per token: a set discount off the price of the same request sent live.

Same API, one more call.

requests.jsonl (one line)
{"custom_id": "q-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "llama-3.1-8b-instruct", "messages": [{"role": "user", "content": "Summarise this ticket: ..."}]}}
Upload and start
curl https://api.peryn.ai/v1/files -H "Authorization: Bearer $PERYN_API_KEY" -F purpose=batch -F file=@requests.jsonlcurl https://api.peryn.ai/v1/batches -H "Authorization: Bearer $PERYN_API_KEY" -H "Content-Type: application/json" \-d '{"input_file_id": "file-...", "endpoint": "/v1/chat/completions", "completion_window": "24h"}'

Batches run on the Standard tier. Your file and its answers are stored encrypted and deleted 7 days after the batch ends, or as soon as you delete them. They are never used for training.