state and answer a set of typed questions about it, returning a probability, a choice, or a score for each. Send these requests to POST https://api.pinference.ai/api/v1/evaluations; evaluation models are rejected on the Chat Completions API.
state. To charge a team, add -H "X-Prime-Team-ID: your-team-id" to each request. See team billing.
Request body
The whole request body can be at most 16 MiB.
Question types
Every question has atype and an optional instructions field that tells the model what to judge. instructions and the criteria descriptions can be strings or JSON values.
Each answer also echoes its
type. Answers can include additional fields such as confidence depending on the model, so read the fields you need and ignore the rest.
Usage and billing
usage.inputTokens and usage.outputTokens report the tokens processed for the request. Evaluation models are billed on input tokens at the price listed in the model catalog. The response includes an X-Inference-Id header you can share with support to trace a request.
Vercel AI SDK compatibility
The endpoint accepts the same request and response format as the Vercel AI Gateway evaluation model protocol, so clients written for that protocol can point at Prime Inference. Prime Inference also serves the gateway’s protocol path,POST https://api.pinference.ai/v4/ai/evaluation-model, which reads the model ID from the ai-model-id request header when the body has no model field.