> ## Documentation Index
> Fetch the complete documentation index at: https://docs.primeintellect.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluations API

> Score, classify, and answer yes/no questions with evaluation models such as Clef

Evaluation models such as [Clef](/inference/models/clef) do not generate text. They read a `state` and answer a set of typed `questions` about it, returning a probability, a choice, or a score for each. Send these requests to `POST https://api.pinference.ai/api/v1/evaluations`; evaluation models are rejected on the [Chat Completions API](/api-reference/inference-chat-completions).

```bash theme={null}
export PRIME_API_KEY="your-api-key"

curl https://api.pinference.ai/api/v1/evaluations \
  -H "Authorization: Bearer $PRIME_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cloudflare/clef-flash",
    "state": "User: the checkout page returns a 500 every time I pay.",
    "questions": {
      "incident": {
        "type": "boolean",
        "instructions": "Is the user reporting an outage?"
      },
      "team": {
        "type": "choice",
        "criteria": {
          "engineering": "Bugs, errors, and outages",
          "billing": "Charges, refunds, and invoices"
        }
      },
      "severity": {
        "type": "score",
        "criteria": ["minor", "major", "critical"]
      }
    }
  }'
```

```json theme={null}
{
  "answers": {
    "incident": {"type": "boolean", "probability": 0.95},
    "team": {"type": "choice", "choice": "engineering", "probabilities": {"engineering": 0.97, "billing": 0.03}},
    "severity": {"type": "score", "score": 1.5}
  },
  "usage": {"inputTokens": 123, "outputTokens": 0},
  "warnings": []
}
```

All questions in a request are answered together against the same `state`. To charge a team, add `-H "X-Prime-Team-ID: your-team-id"` to each request. See [team billing](/inference/overview#team-billing).

## Request body

| Field | Type | Description |
| - | - | - |
| `model` | string | An evaluation model ID from the [models API](/api-reference/inference-models), such as `cloudflare/clef` or `cloudflare/clef-flash`. |
| `state` | string, object, or array | The content to evaluate: a conversation, a document, an agent trace, or any JSON value. |
| `questions` | object | 1 to 64 questions, keyed by an ID you choose. IDs may contain letters, digits, `_`, `.`, and `-`, up to 100 characters. Each answer is returned under the same ID. |
| `images` | array of strings | Optional. Up to 4 images as base64 `data:` URLs (PNG, JPEG, or WebP), at most 4 MiB each and 8 MiB in total. Remote image URLs are not accepted. |

The whole request body can be at most 16 MiB.

## Question types

Every question has a `type` and an optional `instructions` field that tells the model what to judge. `instructions` and the `criteria` descriptions can be strings or JSON values.

| Type | `criteria` | Answer fields |
| - | - | - |
| `boolean` | Optional. An object with `"true"` and/or `"false"` keys that describe what each outcome means. | `probability`: the probability that the answer is true, from 0 to 1. |
| `choice` | Required. An object of 1 to 255 options, mapping each option name to a description (or `null`). | `choice`: the selected option name. May include `probabilities` per option. |
| `score` | Required. An array of 2 to 10 descriptions, ordered from lowest to highest. | `score`: a number on the scale defined by `criteria`. |

Each answer also echoes its `type`. Answers can include additional fields such as `confidence` depending on the model, so read the fields you need and ignore the rest.

## Usage and billing

`usage.inputTokens` and `usage.outputTokens` report the tokens processed for the request. Evaluation models are billed on input tokens at the price listed in the [model catalog](https://app.primeintellect.ai/dashboard/inference?tab=models). The response includes an `X-Inference-Id` header you can share with support to trace a request.

## Vercel AI SDK compatibility

The endpoint accepts the same request and response format as the Vercel AI Gateway evaluation model protocol, so clients written for that protocol can point at Prime Inference. Prime Inference also serves the gateway's protocol path, `POST https://api.pinference.ai/v4/ai/evaluation-model`, which reads the model ID from the `ai-model-id` request header when the body has no `model` field.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.