> ## Documentation Index
> Fetch the complete documentation index at: https://vastai-80aa3a82-auto-cli-sdk-preview-pr-398.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# vastai run benchmarks

Benchmark a template against one or more GPUs

## Usage

```bash theme={null}
vastai run benchmarks [OPTIONS]
```

## Options

<ParamField path="--template_hash" type="string">
  (required, one of --template\_hash or --template\_id) template hash to benchmark
</ParamField>

<ParamField path="--template_id" type="integer">
  (required, one of --template\_hash or --template\_id) template id
</ParamField>

<ParamField path="--gpus" type="string">
  comma-separated GPU names (e.g. RTX\_4090,RTX\_3090); optional Nx prefix takes precedence over --num\_gpus (e.g. "2x RTX\_4090")
</ParamField>

<ParamField path="--num_gpus" type="integer">
  GPUs per instance for tokens without an Nx prefix (default 1); overridden by inline Nx in --gpus
</ParamField>

<ParamField path="--timeout" type="integer" default="3600">
  max seconds to wait for a benchmark before giving up (default 3600)
</ParamField>

<ParamField path="--yes" type="boolean">
  Skip confirmation prompt
</ParamField>

## Description

Rents one instance per GPU in parallel, measures perf, tears down.
Each rental runs for up to --timeout seconds and costs real money.

## Examples

```bash theme={null}
# auto-sweep the default GPUs against TGI
    vastai run benchmarks --template_hash 79ebdd2ebfb9d42cedf7a221c42d37a5

    # specific GPUs against vLLM
    vastai run benchmarks --template_hash 393fa8572e6c73c927c8275fe4dffd53 --gpus RTX_4090,RTX_3090

    # multi-GPU configurations via inline Nx prefix against ComfyUI
    vastai run benchmarks --template_hash 40ef49becc953aa910ee05bd4653b9b3 --gpus "2x RTX_4090, 2x RTX_3090"

    # default count for tokens without an Nx prefix
    vastai run benchmarks --template_hash 79ebdd2ebfb9d42cedf7a221c42d37a5 --gpus RTX_4090,RTX_3090 --num_gpus 2

    # shorter timeout (30 min), skipping the cost prompt
    vastai run benchmarks --template_hash 393fa8572e6c73c927c8275fe4dffd53 --timeout 1800 -y

    # raw JSON output for piping into another tool
    vastai run benchmarks --template_hash 40ef49becc953aa910ee05bd4653b9b3 --raw
```

## Global Options

The following options are available for all commands:

| Option          | Description                                           |
| --------------- | ----------------------------------------------------- |
| `--url URL`     | Server REST API URL                                   |
| `--retry N`     | Retry limit                                           |
| `--raw`         | Output machine-readable JSON                          |
| `--explain`     | Verbose explanation of API calls                      |
| `--api-key KEY` | API key (defaults to `~/.config/vastai/vast_api_key`) |