Prerequisites
[ckpt] block the run leaves you nothing to evaluate:
Step 1: Choose an Environment
You can use an existing environment from the Environments Hub or create your own. For this walkthrough we’ll usealphabet-sort — a multi-turn game where the model must sort letters into alphabetical order. If you’re new to how environments work, see The Environment Model.
Step 2: Run a Baseline Evaluation
Before training, evaluate the base model to establish a baseline. This confirms the environment works and shows you where the model starts:Step 3: Choose a Model and GPU Type
$/GPU-hour and whether a run starts now or queues. Models on that list are cached and start warm. An on-demand run can also train a model that is not listed; it downloads the weights first. See Models.
For a first run, pick the smallest model that exercises your environment end to end. Validate cheaply, then scale.
Step 4: Create a Training Config
Training runs are configured with a prime-rl.toml file. The top-level mode names the run kind, and the [deployment] block sizes the run and is what routes it onto Hosted Training.
[model].name— a Hugging Face model ID or an absolute/volume/...path to weights on your volume. Cached IDs start warm; uncached IDs download at startup on on-demand capacity[ckpt]— save checkpoints. Off by default; with no[ckpt]block nothing is written[deployment]— how many trainer and inference GPUs to reserve; required for dispatchmax_steps— total training steps[orchestrator].batch_size— rollouts consumed per training batch[orchestrator.train].group_size— rollouts generated per dataset example[orchestrator.train.sampling].max_completion_tokens— max tokens the model generates per turn[[orchestrator.train.source]]— one block per training environment, carrying a nestedenvblock
alphabet-sort taskset bundled with the hosted runtime and the null harness. The Hub package used for the baseline evaluation is a separate package; its legacy API is incompatible with the current hosted runtime. For other environments, use a runtime-compatible taskset and harness.
Trace storage and costs
Traces uploaded by platform runs incur Prime Traces ingestion and storage charges, separately from training charges. Review workspace upload controls before launching. See Viewing and deleting traces to manage the resulting data.Step 5: Launch the Training Run
-e / --env-var or --env-file:
Step 6: Monitor the Run
In the terminal — stream logs in real time:-c / --component:
--search, --regex, --level, and --since.
On the dashboard — open the URL printed when the run started, for reward curves, rubric scores, reward distributions, and individual rollouts.
Key metrics to watch:
- Reward — the overall reward curve should trend upward over time
- Rubric — individual rubric component scores
- Reward Distribution — should shift from lower to higher values as training progresses
Step 7: Review Results
Serve the checkpoint on Prime
To query the trained checkpoint through Prime’s OpenAI-compatible API, deploy a training checkpoint. This path loads the saved checkpoint directly and does not require the manual export below. For a run that saves to a volume, wait until the run is completed, stopped, or paused and the final volume sync has finished before deploying.50 with a ready step from the checkpoint list. Use the returned deployment’s model identifier to query it, then delete the inference deployment when you finish to release its GPUs.
Retrieve the checkpoint
The RL run’s checkpoints, logs, and other outputs are saved to the volume when the run ends. Wait for the volume sync to finish, then download the run directory:Export to HF format
For use outside Prime’s checkpoint deployment workflow, convert the DCP-sharded trainer checkpoint to Hugging Face weights. The converter istools/convert_dcp_to_bf16.py in the prime-rl repo. It needs:
- At least one GPU.
- The run’s resolved config. The converter reads model and tokenizer settings from
<run-dir>/configs/latest/resolved/trainer.jsonfor RL orsft.jsonfor SFT. Hosted runs currently do not save this config with the checkpoints. Obtain it from Prime support and place it at the required path before converting. The launch TOML is not a substitute for the resolved JSON config. - Access to the base model and tokenizer. If the resolved config uses
/volume/...paths, make those directories available on the machine doing the conversion.
./run-dir/checkpoints/step_50/weights. That directory loads into an HF consumer. See Exporting Checkpoints for FP8 variants.
Hosted SFT runs keep the same layout under
runs/<runId>/. Obtain the resolved config before running the converter.Evaluate against your baseline
Serve the exported weights, then point the Step 2 evaluation at them. This evaluation runs locally, so install the environment first:--client.base_url points the eval client at your endpoint (see prime-rl evaluation). Run it from a prime-rl checkout — installing the Prime CLI alone does not provide uv run eval. Compare the reward against the baseline you recorded in Step 2.
Metrics, samples, and progress are persisted to the dashboard either way, but trainer, inference, and env-server pod logs go away with the pods when the run is torn down — pull anything you need while the run is live.
Run Size Guidelines
Each of these omits the blocks already covered above. Add[ckpt] and dispatch with --volume to keep weights. See Checkpoint settings for retention settings.
Small (validation)
Verify your environment and config work before committing to a longer run.Medium (experimentation)
Good for iterating on environment design and hyperparameters.Large (production)
WANDB_API_KEY is required for the [wandb] block to take effect — without it the block is stripped and the run trains without W&B logging.Hosted RL
Config reference, dispatch knobs, limits, and run lifecycle.
Billing
How GPU-hours are metered and what gets billed.
Troubleshooting
Solutions for common issues with Hosted Training runs.