Skip to main content
Volumes are in closed beta for full fine-tuning. Reach out to us to get access.
A volume is named storage for full fine-tuning outputs. It is a CephFS-backed persistent volume claim (PVC) in your team’s or personal namespace on a cluster. It outlives individual runs, so outputs stay on the volume after a run ends. Volumes default to 1 TiB (1Ti) and can grow to 10 TiB (10Ti). They cannot shrink.

​
Before you start

Install and authenticate the Prime CLI. These commands require Prime CLI 0.9.0 or later. Volumes use the team configured in your CLI. Select a team before creating a volume:
With no team configured, volumes belong to your personal account. To switch back to personal volumes:
Use the same team or personal account when creating, listing, resizing, deleting, and training with a volume. See CLI configuration.

​
Create a volume

Choose a name using lowercase letters, digits, and hyphens:
This creates a 1Ti volume. To choose a different starting size, pass --size:
To choose a cluster, add --cluster <name>; otherwise, the CLI uses your first available cluster.

​
List volumes

The table shows each volume’s name, size, cluster, status, and creation time. For JSON output:
Both create and list accept --output json (or -o json). JSON includes the volume’s clusterId.

​
SSH into a volume

Open a session to inspect outputs or stage datasets. Configure your SSH key with prime config set-ssh-key-path <private-key-path> first.
The CLI opens a session, waits for the connection, pins the session’s SSH host key, and connects. The volume is mounted at /volume. Bare prime volumes ssh <name> opens a read-write session by default; --read-only (or --read) mounts it read-only to inspect or download files without changing them. Pass --read-write explicitly to upload or edit files:
Sessions work from your laptop over the public internet — no VPN or special network access is needed. Copy files with prime volumes get <name> <remote> [local] or prime volumes put <name> <local> [remote]:
These commands open or reuse a session and use rsync when available, or scp otherwise. Downloads (get) work with either session mode; uploads (put) require read-write access, since writes fail on a read-only mount. get requests a read-only session but can reuse your read-write session; put opens or reuses a read-write session. The default local destination for get is .; the default remote destination for put is /. Remote paths are relative to the volume root: /datasets/my-data in put is /volume/datasets/my-data inside SSH. Stop a session using the ID printed when it opens:

​
Use a volume for training

Pass a volume name when launching a full fine-tuning run:
Use a full fine-tuning config with a [deployment] block. Volumes are not supported for LoRA runs. If the volume is missing from the selected team or personal account, the CLI creates it (default 1Ti), waits until it is RUNNING, then dispatches the run. Set --volume-size to choose its initial size; existing volumes are left unchanged. The run writes its outputs under runs/<runId>/ on the volume, giving each run its own directory. A config with a [data] block and no [trainer] or [orchestrator] block auto-dispatches as Hosted SFT, mounting the volume read-only at /volume for data.name to read the staged dataset. The volume pins the run to the cluster where it was created. That cluster must support the run’s GPU requirements and have the base model cached — unless the run is Hosted SFT reading its model from the volume itself (see below); the run cannot move to another cluster just because it has free GPUs. You can also set the volume at the top level of your TOML config, before any tables:
The --volume and --volume-size flags take precedence over their config values.

​
Train from weights already on the volume

Hosted SFT runs can load their base model from the volume instead of the cluster model cache — start from weights you staged yourself, or continue a previous run’s checkpoint via [resume].dir (see Volume-hosted models). Point [model].name at the model directory using its absolute mount path. Unlike data.name, the model path must start with /volume/:
Dispatch as usual with the volume attached:
Requirements and behavior:
  • The run must carry the volume (--volume or volume = "..." in the config); otherwise dispatch is rejected.
  • The path needs at least one directory after /volume/, and each segment starts with a letter or digit, then letters, digits, ., -, or _ only — . and .. are rejected.
  • The directory holds Hugging Face-format weights (config.json plus safetensors files) — something you copied up with prime volumes put. A previous run’s checkpoint does not qualify (it is in prime-rl’s distributed format): continue those with [resume].dir instead (Volume-hosted models).
  • The model does not need to be in the cluster model cache. No cache check applies — dispatch only requires the volume’s cluster to fit the run’s GPU requirements.
  • The volume stays read-only on the trainer. Any weight conversion the trainer performs is written to the run’s own scratch space, never back onto the volume.
RL runs ([trainer]/[orchestrator] configs) cannot read their model from a volume yet — they still require a cached HF model id. For staging weights, chaining runs from your own checkpoints, and how dispatch changes, see the dedicated guide: Volume-hosted models.

​
Grow a volume

Pass a new size larger than the current size, up to 10Ti:
The volume grows in place. Running pods see the new size; you do not need to stop the run. Shrinking is not supported.

​
Delete a volume

Deleting a volume deletes all data on it, including outputs from every run that used it.
The CLI asks for confirmation. Pass --yes (or -y) to skip the prompt. Team volumes require a team admin to delete them; personal volumes can be deleted by their owner. Deletion is refused while the volume is still provisioning or runs are still mounting it. Wait for those runs to finish, or stop them and wait for their pods to terminate, then retry. --yes skips only the confirmation; it does not bypass this guard.