Skip to main content
This guide walks you through a complete Hosted Training run — choosing an environment, establishing a baseline, writing a config, dispatching the run, monitoring it, and reviewing results. For the config schema and dispatch options in reference form, see Hosted RL.

Prerequisites

You also need on-demand training enabled and a funded wallet — see Billing. Start from Getting Started if you haven’t run anything yet. Create a volume to hold the trained weights. Checkpointing is off by default and pods are torn down when a run ends, so without a volume and a [ckpt] block the run leaves you nothing to evaluate:

Step 1: Choose an Environment

You can use an existing environment from the Environments Hub or create your own. For this walkthrough we’ll use alphabet-sort — a multi-turn game where the model must sort letters into alphabetical order. If you’re new to how environments work, see The Environment Model.

Step 2: Run a Baseline Evaluation

Before training, evaluate the base model to establish a baseline. This confirms the environment works and shows you where the model starts:
A good training environment should have a baseline reward between roughly 10–80%. If the model scores 0% after many attempts, the task is too hard. If it’s already at 80%+, consider harder examples or a different environment.
The evaluation runs on the platform, so there’s nothing to install locally. The command prints an evaluation ID and a dashboard link where the results appear. To stream its logs:
See Evaluating environments for the full set of hosted evaluation options.

Step 3: Choose a Model and GPU Type

This lists the models you can dispatch, plus an On-Demand Capacity table of purchasable GPU types with their current $/GPU-hour and whether a run starts now or queues. Models on that list are cached and start warm. An on-demand run can also train a model that is not listed; it downloads the weights first. See Models. For a first run, pick the smallest model that exercises your environment end to end. Validate cheaply, then scale.

Step 4: Create a Training Config

Training runs are configured with a prime-rl .toml file. The top-level mode names the run kind, and the [deployment] block sizes the run and is what routes it onto Hosted Training.
The key fields:
  • [model].name — a Hugging Face model ID or an absolute /volume/... path to weights on your volume. Cached IDs start warm; uncached IDs download at startup on on-demand capacity
  • [ckpt] — save checkpoints. Off by default; with no [ckpt] block nothing is written
  • [deployment] — how many trainer and inference GPUs to reserve; required for dispatch
  • max_steps — total training steps
  • [orchestrator].batch_size — rollouts consumed per training batch
  • [orchestrator.train].group_size — rollouts generated per dataset example
  • [orchestrator.train.sampling].max_completion_tokens — max tokens the model generates per turn
  • [[orchestrator.train.source]] — one block per training environment, carrying a nested env block
This config uses the native alphabet-sort taskset bundled with the hosted runtime and the null harness. The Hub package used for the baseline evaluation is a separate package; its legacy API is incompatible with the current hosted runtime. For other environments, use a runtime-compatible taskset and harness.

Trace storage and costs

Traces uploaded by platform runs incur Prime Traces ingestion and storage charges, separately from training charges. Review workspace upload controls before launching. See Viewing and deleting traces to manage the resulting data.

Step 5: Launch the Training Run

The CLI confirms the configuration, echoes the GPU type and hourly rate it resolved, and prints a dashboard link:
Pass credentials your environment needs with -e / --env-var or --env-file:
Config errors — bad schema, an oversized deployment, or a model that a reserved cluster does not have cached — fail the dispatch immediately with a reason, before any GPU is reserved and before anything bills.

Step 6: Monitor the Run

In the terminal — stream logs in real time:
A run has several components; pick one with -c / --component:
Filter with --search, --regex, --level, and --since. On the dashboard — open the URL printed when the run started, for reward curves, rubric scores, reward distributions, and individual rollouts. Key metrics to watch:
  • Reward — the overall reward curve should trend upward over time
  • Rubric — individual rubric component scores
  • Reward Distribution — should shift from lower to higher values as training progresses
Stop a run early to release its GPUs and stop the meter:
To release the GPUs but continue from the current step later, pause the run instead.

Step 7: Review Results

Serve the checkpoint on Prime

To query the trained checkpoint through Prime’s OpenAI-compatible API, deploy a training checkpoint. This path loads the saved checkpoint directly and does not require the manual export below. For a run that saves to a volume, wait until the run is completed, stopped, or paused and the final volume sync has finished before deploying.
Replace 50 with a ready step from the checkpoint list. Use the returned deployment’s model identifier to query it, then delete the inference deployment when you finish to release its GPUs.

Retrieve the checkpoint

The RL run’s checkpoints, logs, and other outputs are saved to the volume when the run ends. Wait for the volume sync to finish, then download the run directory:

Export to HF format

For use outside Prime’s checkpoint deployment workflow, convert the DCP-sharded trainer checkpoint to Hugging Face weights. The converter is tools/convert_dcp_to_bf16.py in the prime-rl repo. It needs:
  • At least one GPU.
  • The run’s resolved config. The converter reads model and tokenizer settings from <run-dir>/configs/latest/resolved/trainer.json for RL or sft.json for SFT. Hosted runs currently do not save this config with the checkpoints. Obtain it from Prime support and place it at the required path before converting. The launch TOML is not a substitute for the resolved JSON config.
  • Access to the base model and tokenizer. If the resolved config uses /volume/... paths, make those directories available on the machine doing the conversion.
Run the converter from a prime-rl checkout on a machine with a GPU:
This writes sharded safetensors plus config and tokenizer assets to ./run-dir/checkpoints/step_50/weights. That directory loads into an HF consumer. See Exporting Checkpoints for FP8 variants.
Hosted SFT runs keep the same layout under runs/<runId>/. Obtain the resolved config before running the converter.

Evaluate against your baseline

Serve the exported weights, then point the Step 2 evaluation at them. This evaluation runs locally, so install the environment first:
--client.base_url points the eval client at your endpoint (see prime-rl evaluation). Run it from a prime-rl checkout — installing the Prime CLI alone does not provide uv run eval. Compare the reward against the baseline you recorded in Step 2.
Metrics, samples, and progress are persisted to the dashboard either way, but trainer, inference, and env-server pod logs go away with the pods when the run is torn down — pull anything you need while the run is live.

Run Size Guidelines

Each of these omits the blocks already covered above. Add [ckpt] and dispatch with --volume to keep weights. See Checkpoint settings for retention settings.

Small (validation)

Verify your environment and config work before committing to a longer run.

Medium (experimentation)

Good for iterating on environment design and hyperparameters.

Large (production)

WANDB_API_KEY is required for the [wandb] block to take effect — without it the block is stripped and the run trains without W&B logging.

Hosted RL

Config reference, dispatch knobs, limits, and run lifecycle.

Billing

How GPU-hours are metered and what gets billed.

Troubleshooting

Solutions for common issues with Hosted Training runs.