Create Evaluation
Create a new evaluation
This endpoint supports:
- Environment evaluations: Provide environments
- Prime RL evaluations: Provide run_id
- Suite evaluations: Provide suite_id
Ownership:
- If team_id is provided in request, the evaluation will be owned by the team
- Otherwise, the evaluation will be owned by the authenticated user
Authorizations
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
Body
Request to create a new evaluation
Name of the evaluation
Team ID if creating evaluation for a team
List of environment references with optional version IDs
Suite ID if this evaluation is part of a suite
Run ID for Prime RL runs (optional)
Whether this is a hosted evaluation
Prime Inference model ID
Hosted evaluation configuration
Model name
Dataset name
Framework used (e.g., 'prime-rl', 'openai/evals')
Type of task (e.g., 'classification', 'generation')
Description of the evaluation
Tags for categorization
Additional metadata
High-level metrics summary
Whether this evaluation is publicly shareable by link
Whether this public evaluation appears on environment leaderboards
Response
Successful Response
Response after creating an evaluation
ID of the created evaluation
Evaluation status enum
PENDING, RUNNING, PROCESSING, COMPLETED, FAILED, TIMEOUT, CANCELLED Type: 'prime_rl', 'environment', or 'suite'
suite, training, environment