Env

class Env(ABC):
    @abstractmethod
    async def run(self, task: Task, agents: Agents) -> None:
        """Run a single multi-agent episode."""
        ...

• IsolatedVerifierEnv runs one solver, transfers only declared artifacts into a fresh configured runtime, and runs deterministic task scoring there.
• The AgenticJudgeEnv defines the sequential interaction between a solver and judge agent. The judge can re-use the same runtime after the solver (SharedAgenticJudgeEnv) or use its own, new runtime IsolatedAgenticJudgeEnv.
• The UserSimEnv models users as agents, and the episode is a turn-by-turn conversation between the user and assistant agents.
• The BestOfNEnv runs n independent attempts at the same task, then marks which attempt achieved the highest reward (best) and whether any attempt crossed a success threshold (pass_at_n), which is useful for rejection sampling and pass@k evaluation.

​

Isolated deterministic verification

uv run vf-eval my-task --env.id isolated-verifier --env.agent.runtime.type docker

from pathlib import Path

import verifiers.v1 as vf


PRIVATE_TESTS = Path("tests/test_solution.sh").read_bytes()


class CodeTask(vf.Task[CodeData]):
    async def stage_verifier(self, runtime: vf.Runtime) -> None:
        await runtime.write("/tmp/test_solution.sh", PRIVATE_TESTS)

    @vf.reward
    async def tests(self, runtime: vf.Runtime) -> float:
        result = await runtime.run(["bash", "/tmp/test_solution.sh"], {})
        return float(result.exit_code == 0)


task = CodeTask(
    CodeData(
        prompt="Fix the implementation.",
        artifacts=[vf.Artifact(source="src")],
    )
)

1. The task and harness run normally, including task finalize and harness metrics, but task metrics and rewards are deferred.
2. The environment collects the declared paths and /logs/artifacts, then destroys the solver runtime.
3. It creates a fresh task controller and provisions either the same resolved container/runtime policy or the independently configured verifier runtime, runs task setup, restores the artifacts at their original paths, runs task stage_verifier, reapplies the execution network policy, and runs task metrics and rewards onto the solver trace.