Check the prerequisites
- The run belongs to a team you can access.
- The checkpoint is saved and available as
READY. Only complete checkpoints on a finalized volume can be served, so the run must be completed, stopped, or paused, and its final volume sync must have finished. Checkpoints still on an actively training run’s volume are not available. Pause a running run to sync its volume and release its GPUs without waiting for it to finish. - The run’s base model has a serving configuration in the inference catalog. If deployment reports that the base model is missing, ask support to add it.
- On-demand deployment needs a funded team wallet and available capacity. Using the run’s own cluster needs an admin of that cluster’s team, with inference and self-service actions enabled.
factory:read and factory:write for deployment management, plus Inference permission to query the endpoint.
Deploy in the dashboard
- Select the run’s team, then open Dedicated Endpoints.
- Click Create endpoint, and select Custom weights.
- Search for the training run, then choose one of its saved checkpoint steps.
- Under Deploy to, choose the cluster the run trained on or On-demand capacity when both are offered. Review the cost before using on-demand GPUs.
- Click Create endpoint, then wait for it to be running.
Deploy with the CLI
List ready checkpoints for your run:100 with an available step. The command has no default checkpoint step. The equivalent training command is:
onDemand field.
If that checkpoint already has a pending, deploying, or running deployment, the request returns the existing deployment, even when you ask for a different placement or replica count. Deploying another step creates a separate deployment. Stop the old one when you finish comparing them.
Query and manage the endpoint
Inspect the deployment returned by the command:RUNNING, use servedModelAlias as your request’s model identifier and endpointUrl as the OpenAI-compatible base URL. Use the run’s team context for requests. See Send a request for CLI and Python examples.
The endpoint has its own lifecycle. Exiting the CLI does not stop it. Manage replicas and release the GPUs with:
The checkpoint is no longer available. Deleting the inference deployment leaves the training run and source checkpoint intact.
If no checkpoints appear, see Checkpoint unavailable. To export weights for use outside Prime, follow Retrieve and export the checkpoint.