> ## Documentation Index
> Fetch the complete documentation index at: https://docs.primeintellect.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Volumes (Beta)

> Keep full fine-tuning outputs across runs with named volumes

<Note>
  Volumes are in **closed beta** for full fine-tuning. Reach out to us to get access.
</Note>

A volume is named storage for [full fine-tuning](/hosted-training/full-finetuning) outputs. It is a CephFS-backed persistent volume claim (PVC) in your team's or personal namespace on a cluster. It outlives individual runs, so outputs stay on the volume after a run ends.

Volumes default to **1 TiB** (`1Ti`) and can grow to **10 TiB** (`10Ti`). They cannot shrink.

## Before you start

[Install and authenticate the Prime CLI](/cli-reference/introduction). These commands require Prime CLI 0.9.0 or later.

Volumes use the team configured in your CLI. Select a team before creating a volume:

```bash theme={null}
prime teams list
prime config set-team-id <team-id>
```

With no team configured, volumes belong to your personal account. To switch back to personal volumes:

```bash theme={null}
prime config remove-team-id
```

Use the same team or personal account when creating, listing, resizing, deleting, and training with a volume. See [CLI configuration](/cli-reference/config-cli#team-settings).

## Create a volume

Choose a name using lowercase letters, digits, and hyphens:

```bash theme={null}
prime volumes create training-outputs
```

This creates a `1Ti` volume. To choose a different starting size, pass `--size`:

```bash theme={null}
prime volumes create experiment-outputs --size 500Gi
```

To choose a cluster, add `--cluster <name>`; otherwise, the CLI uses your first available cluster.

## List volumes

```bash theme={null}
prime volumes list
```

The table shows each volume's name, size, cluster, status, and creation time. For JSON output:

```bash theme={null}
prime volumes list --output json
```

Both `create` and `list` accept `--output json` (or `-o json`). JSON includes the volume's `clusterId`.

## SSH into a volume

Open a session to inspect outputs or stage datasets. Configure your SSH key with `prime config set-ssh-key-path <private-key-path>` first.

```bash theme={null}
prime volumes ssh training-outputs --read-only
```

The CLI opens a session, waits for the connection, pins the session's SSH host key, and connects. The volume is mounted at `/volume`. Bare `prime volumes ssh <name>` opens a read-write session by default; `--read-only` (or `--read`) mounts it read-only to inspect or download files without changing them. Pass `--read-write` explicitly to upload or edit files:

```bash theme={null}
prime volumes ssh training-outputs --read-write
```

Sessions work from your laptop over the public internet — no VPN or special network access is needed.

Copy files with `prime volumes get <name> <remote> [local]` or `prime volumes put <name> <local> [remote]`:

```bash theme={null}
prime volumes get training-outputs /runs/<runId>/ ./outputs
prime volumes put training-outputs ./my-data /datasets/my-data
```

These commands open or reuse a session and use rsync when available, or scp otherwise. Downloads (`get`) work with either session mode; uploads (`put`) require read-write access, since writes fail on a read-only mount. `get` requests a read-only session but can reuse your read-write session; `put` opens or reuses a read-write session. The default local destination for `get` is `.`; the default remote destination for `put` is `/`. Remote paths are relative to the volume root: `/datasets/my-data` in `put` is `/volume/datasets/my-data` inside SSH.

Stop a session using the ID printed when it opens:

```bash theme={null}
prime volumes stop training-outputs <session_id>
```

## Use a volume for training

Pass a volume name when launching a full fine-tuning run:

```bash theme={null}
prime train config.toml --volume training-outputs
```

Use a [full fine-tuning config](/hosted-training/full-finetuning#config) with a `[deployment]` block. Volumes are not supported for LoRA runs.

If the volume is missing from the selected team or personal account, the CLI creates it (default `1Ti`), waits until it is `RUNNING`, then dispatches the run. Set `--volume-size` to choose its initial size; existing volumes are left unchanged. The run writes its outputs under `runs/<runId>/` on the volume, giving each run its own directory.

A config with a `[data]` block and no `[trainer]` or `[orchestrator]` block auto-dispatches as [Hosted SFT](/hosted-training/hosted-sft), mounting the volume read-only at `/volume` for `data.name` to read the staged dataset.

The volume pins the run to the cluster where it was created. That cluster must support the run's GPU requirements and have the base model cached — unless the run is Hosted SFT reading its model from the volume itself (see below); the run cannot move to another cluster just because it has free GPUs.

You can also set the volume at the top level of your TOML config, before any tables:

```toml theme={null}
volume = "training-outputs"
volume_size = "1Ti" # Used only when creating a missing volume
```

The `--volume` and `--volume-size` flags take precedence over their config values.

### Train from weights already on the volume

Hosted SFT runs can load their base model from the volume instead of the cluster model cache — start from weights you staged yourself, or continue a previous run's checkpoint via `[resume].dir` (see [Volume-hosted models](/hosted-training/volume-models#chaining-from-a-previous-runs-checkpoint)).

Point `[model].name` at the model directory using its absolute mount path. Unlike `data.name`, the model path must start with `/volume/`:

```toml theme={null}
[ckpt]

[model]
name = "/volume/models/glm-5.3-hf"  # weights already on the volume

[data]
name = "datasets/my-dataset"         # dataset on the same volume
type = "sft"
seq_len = 4096
```

Dispatch as usual with the volume attached:

```bash theme={null}
prime train sft.toml --volume research
```

Requirements and behavior:

* The run must carry the volume (`--volume` or `volume = "..."` in the config); otherwise dispatch is rejected.
* The path needs at least one directory after `/volume/`, and each segment starts with a letter or digit, then letters, digits, `.`, `-`, or `_` only — `.` and `..` are rejected.
* The directory holds Hugging Face-format weights (`config.json` plus safetensors files) — something you copied up with `prime volumes put`. A previous run's checkpoint does **not** qualify (it is in prime-rl's distributed format): continue those with `[resume].dir` instead ([Volume-hosted models](/hosted-training/volume-models#chaining-from-a-previous-runs-checkpoint)).
* The model does not need to be in the cluster model cache. No cache check applies — dispatch only requires the volume's cluster to fit the run's GPU requirements.
* The volume stays read-only on the trainer. Any weight conversion the trainer performs is written to the run's own scratch space, never back onto the volume.

RL runs (`[trainer]`/`[orchestrator]` configs) cannot read their model from a volume yet — they still require a cached HF model id.

For staging weights, chaining runs from your own checkpoints, and how dispatch changes, see the dedicated guide: [Volume-hosted models](/hosted-training/volume-models).

## Grow a volume

Pass a new size larger than the current size, up to `10Ti`:

```bash theme={null}
prime volumes resize training-outputs --size 2Ti
```

The volume grows in place. Running pods see the new size; you do not need to stop the run. Shrinking is not supported.

## Delete a volume

<Warning>
  Deleting a volume deletes all data on it, including outputs from every run that used it.
</Warning>

```bash theme={null}
prime volumes delete training-outputs
```

The CLI asks for confirmation. Pass `--yes` (or `-y`) to skip the prompt.

Team volumes require a team admin to delete them; personal volumes can be deleted by their owner.

Deletion is refused while the volume is still provisioning or runs are still mounting it. Wait for those runs to finish, or [stop them](/hosted-training/full-finetuning#run-lifecycle) and wait for their pods to terminate, then retry. `--yes` skips only the confirmation; it does not bypass this guard.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.