Skip to main content
Create a dedicated endpoint when you need GPUs reserved for your model. You can use Prime’s on-demand capacity or a Model Factory cluster your team owns. To serve trained weights, use Deploy a training checkpoint.

Before you start

  • For on-demand capacity, fund the personal or team wallet that owns the deployment. Review the GPU type, hourly price, and availability in the creation dialog.
  • For a team catalog deployment, use a team admin account. For a team cluster, inference and self-service actions must be enabled on that cluster.
  • For CLI operations, install Prime CLI 0.21.0 or later and authenticate. Your API key needs Model Factory read and write permissions, factory:read and factory:write. Requests to the deployed model need the Inference permission.
On-demand deployments bill reserved GPUs per GPU-hour from the moment the endpoint reaches RUNNING, including time with no requests. Closing your terminal does not stop the deployment. Deployments on your team’s cluster use that cluster’s GPUs without an additional on-demand GPU-hour charge. An on-demand endpoint stops automatically when the wallet that pays for it runs out of funds. Its error message says so; top up the wallet and deploy again.

Create an endpoint in the dashboard

  1. Open Dedicated Endpoints, and select the personal or team account that will own the endpoint.
  2. Click Create endpoint.
  3. Select Open weights, then choose a model from the catalog.
  4. Under Deploy to, select On-demand capacity, which lets Prime choose the cluster, or one of your team’s clusters.
  5. Set the deployment name and replica count. Each replica is a complete serving engine and can require several GPUs.
  6. Review the placement, GPU reservation, and projected cost, then click Create endpoint.
The endpoint page shows startup progress in the endpoint’s status. Wait until the endpoint is running before sending requests. Available GPUs do not mean the model is already loaded. Startup can also include preparing workers and loading weights. A running endpoint also reports serving metrics and repeats the request example on its API tab.

Create an endpoint with the CLI

List the deployable models:
Copy a value from the ID column. This catalog ID selects a deployment configuration. It differs from the Hugging Face repository and the model identifier used for requests. Create an on-demand endpoint in your selected account:
For a team deployment, select the team explicitly:
To use a team cluster, copy its ID from the catalog’s Team Clusters table and add --cluster:
The command prints a deployment ID. With --wait, it polls until the deployment reaches RUNNING or a terminal state. It then prints the model identifier, endpoint, and a request example. If the wait times out, check the existing deployment before creating another one:
A timeout stops the local wait. The deployment can continue starting and later incur charges.

Send a request

Copy servedModelAlias and endpointUrl from the deployment details:
Set these values in your environment. endpointUrl is the OpenAI-compatible base URL and is available once the deployment is RUNNING. It is the same Prime Inference base URL that serves shared models; the model field selects your deployment.
Use the returned alias exactly. The deployment name, catalog ID, and base model repository do not select your dedicated endpoint.
For a team endpoint, first select its team in CLI configuration with prime config set-team-id. Direct HTTP and SDK requests instead need X-Prime-Team-ID on each request. For a direct HTTP request:
For a team endpoint, add -H "X-Prime-Team-ID: <team-id>" to that command. With the OpenAI Python SDK:
For a personal endpoint, use the owner’s API key and omit the team header. See deployment troubleshooting if requests fail.

Change the replica count

In the dashboard, open the running endpoint, set the count with the replica controls under Replicas, and click Update. Review the additional GPU cost when scaling up on-demand capacity. Scale a running deployment with the CLI:
The returned replicas is the desired count. Check scaleState while workers converge to that count. The deployment’s maxReplicas, available GPUs, and account limits can restrict scaling. On-demand scale-ups reserve and bill the added GPUs. To release all GPUs, delete the deployment. Scaling to zero is not supported.

Stop serving and release the GPUs

In the dashboard, open the endpoint’s actions menu, choose Delete, and confirm Delete endpoint. List deployments, or pass --team to list a team’s deployments:
In Prime CLI 0.21.0, omitting --team lists your personal deployments even when CLI configuration has a selected team. Delete the endpoint when you finish:
Confirm the deletion. The API accepts teardown asynchronously and returns TERMINATING. Requests to its alias start failing as the endpoint stops serving, and its GPUs are released. Refresh the Dedicated Endpoints page to check completion. A team admin or the deployment’s creator, while still a team member, can delete a team deployment. For automation and option details, see the CLI reference and deployment API reference.