Before you start
- For on-demand capacity, fund the personal or team wallet that owns the deployment. Review the GPU type, hourly price, and availability in the creation dialog.
- For a team catalog deployment, use a team admin account. For a team cluster, inference and self-service actions must be enabled on that cluster.
- For CLI operations, install Prime CLI 0.21.0 or later and authenticate. Your API key needs Model Factory read and write permissions,
factory:readandfactory:write. Requests to the deployed model need the Inference permission.
RUNNING, including time with no requests. Closing your terminal does not stop the deployment. Deployments on your team’s cluster use that cluster’s GPUs without an additional on-demand GPU-hour charge.
An on-demand endpoint stops automatically when the wallet that pays for it runs out of funds. Its error message says so; top up the wallet and deploy again.
Create an endpoint in the dashboard
- Open Dedicated Endpoints, and select the personal or team account that will own the endpoint.
- Click Create endpoint.
- Select Open weights, then choose a model from the catalog.
- Under Deploy to, select On-demand capacity, which lets Prime choose the cluster, or one of your team’s clusters.
- Set the deployment name and replica count. Each replica is a complete serving engine and can require several GPUs.
- Review the placement, GPU reservation, and projected cost, then click Create endpoint.
Create an endpoint with the CLI
List the deployable models:--cluster:
--wait, it polls until the deployment reaches RUNNING or a terminal state. It then prints the model identifier, endpoint, and a request example.
If the wait times out, check the existing deployment before creating another one:
Send a request
CopyservedModelAlias and endpointUrl from the deployment details:
endpointUrl is the OpenAI-compatible base URL and is available once the deployment is RUNNING. It is the same Prime Inference base URL that serves shared models; the model field selects your deployment.
prime config set-team-id. Direct HTTP and SDK requests instead need X-Prime-Team-ID on each request.
For a direct HTTP request:
-H "X-Prime-Team-ID: <team-id>" to that command. With the OpenAI Python SDK:
Change the replica count
In the dashboard, open the running endpoint, set the count with the replica controls under Replicas, and click Update. Review the additional GPU cost when scaling up on-demand capacity. Scale a running deployment with the CLI:replicas is the desired count. Check scaleState while workers converge to that count. The deployment’s maxReplicas, available GPUs, and account limits can restrict scaling.
On-demand scale-ups reserve and bill the added GPUs. To release all GPUs, delete the deployment. Scaling to zero is not supported.
Stop serving and release the GPUs
In the dashboard, open the endpoint’s actions menu, choose Delete, and confirm Delete endpoint. List deployments, or pass--team to list a team’s deployments:
--team lists your personal deployments even when CLI configuration has a selected team.
Delete the endpoint when you finish:
TERMINATING. Requests to its alias start failing as the endpoint stops serving, and its GPUs are released. Refresh the Dedicated Endpoints page to check completion. A team admin or the deployment’s creator, while still a team member, can delete a team deployment.
For automation and option details, see the CLI reference and deployment API reference.