vastai run benchmarks

Benchmark a template against one or more GPUs

Usage

vastai run benchmarks [OPTIONS]

Options

--template_hash · string

(required, one of —template_hash or —template_id) template hash to benchmark

--template_id · integer

(required, one of —template_hash or —template_id) template id

--gpus · string

comma-separated GPU names (e.g. RTX_4090,RTX_3090); optional Nx prefix takes precedence over —num_gpus (e.g. “2x RTX_4090”)

--num_gpus · integer

GPUs per instance for tokens without an Nx prefix (default 1); overridden by inline Nx in —gpus

--timeout · integer · default: "3600"

max seconds to wait for a benchmark before giving up (default 3600)

--yes · boolean

Skip confirmation prompt

Description

Rents one instance per GPU in parallel, measures perf, tears down. Each rental runs for up to —timeout seconds and costs real money.

Examples

# auto-sweep the default GPUs against TGI
    vastai run benchmarks --template_hash 79ebdd2ebfb9d42cedf7a221c42d37a5

    # specific GPUs against vLLM
    vastai run benchmarks --template_hash 393fa8572e6c73c927c8275fe4dffd53 --gpus RTX_4090,RTX_3090

    # multi-GPU configurations via inline Nx prefix against ComfyUI
    vastai run benchmarks --template_hash 40ef49becc953aa910ee05bd4653b9b3 --gpus "2x RTX_4090, 2x RTX_3090"

    # default count for tokens without an Nx prefix
    vastai run benchmarks --template_hash 79ebdd2ebfb9d42cedf7a221c42d37a5 --gpus RTX_4090,RTX_3090 --num_gpus 2

    # shorter timeout (30 min), skipping the cost prompt
    vastai run benchmarks --template_hash 393fa8572e6c73c927c8275fe4dffd53 --timeout 1800 -y

    # raw JSON output for piping into another tool
    vastai run benchmarks --template_hash 40ef49becc953aa910ee05bd4653b9b3 --raw

Global Options

The following options are available for all commands:

Option	Description
--url URL	Server REST API URL
--retry N	Retry limit
--raw	Output machine-readable JSON
--explain	Verbose explanation of API calls
--api-key KEY	API key (defaults to ~/.config/vastai/vast_api_key)