Verifying Instance Performance

Use these checks after launching an instance to confirm you’re getting the performance you’re paying for, and before opening a support ticket about slow workloads. For the hardware specs to compare against (memory, bandwidth, interconnect), see the GPU comparison table.

Quick health check

# GPUs visible, correct model and count, driver loaded
nvidia-smi

# Detailed per-GPU state: clocks, power limits, ECC errors
nvidia-smi -q

Confirm that:

The GPU model and count match what you rented (H100 80GB, H200 141GB, or B200 192GB)
ECC error counts are zero or not climbing
No GPU is stuck at low clocks or in a fallback power state while idle

GPU diagnostics (DCGM)

NVIDIA’s Data Center GPU Manager runs structured hardware diagnostics:

# Install if not present (Ubuntu)
sudo apt-get install -y datacenter-gpu-manager
sudo systemctl start nvidia-dcgm

# Level 2 diagnostic (~2 minutes): PCIe, memory, compute sanity
dcgmi diag -r 2

# Level 3 (~30 minutes): adds stress and stability tests
dcgmi diag -r 3

Any test reported as Fail is grounds for a support ticket — include the full output.

Compute throughput

A large matrix multiplication measures real achievable throughput:

import torch, time

n = 8192
a = torch.randn(n, n, dtype=torch.bfloat16, device="cuda")
b = torch.randn(n, n, dtype=torch.bfloat16, device="cuda")

for _ in range(10):  # warmup
    a @ b
torch.cuda.synchronize()

iters = 100
start = time.time()
for _ in range(iters):
    a @ b
torch.cuda.synchronize()
elapsed = time.time() - start

tflops = (2 * n**3 * iters) / elapsed / 1e12
print(f"{tflops:.0f} TFLOPS (BF16)")

Compare against the GPU’s datasheet peak for dense BF16. Well-tuned large matmuls should reach a substantial fraction of peak; results dramatically below that across repeated runs (rule out thermal throttling and shared load first) are worth reporting to support.

Memory bandwidth

git clone https://github.com/NVIDIA/nvbandwidth && cd nvbandwidth
sudo apt-get install -y libboost-program-options-dev
cmake . && make

# Host-to-device, device-to-host, and device-to-device bandwidth
./nvbandwidth

Device-to-device results should approach the memory bandwidth listed for your GPU (3.35 TB/s H100, 4.8 TB/s H200, 8 TB/s B200).

Multi-GPU and interconnect

For multi-GPU instances and InfiniBand clusters, verify collective bandwidth with nccl-tests:

git clone https://github.com/NVIDIA/nccl-tests && cd nccl-tests
make

# All-reduce across all 8 GPUs, 8 B to 8 GB message sizes
./build/all_reduce_perf -b 8 -e 8G -f 2 -g 8

Check the busbw column at large message sizes:

On NVLink-connected nodes, bus bandwidth should scale toward the NVLink spec (900 GB/s on H100/H200, 1.8 TB/s on B200) — several hundred GB/s at large sizes is healthy on 8x H100.
On InfiniBand clusters, run the multi-node variant (via MPI) and confirm inter-node bandwidth is consistent with the cluster’s InfiniBand configuration (up to 3.2 Tb/s).

You can also confirm the interconnect topology directly:

nvidia-smi topo -m   # NV-links between GPU pairs
ibstat               # InfiniBand link state and rate (clusters)

Storage I/O

sudo apt-get install -y fio

# Sequential write/read throughput on the current volume
fio --name=seqwrite --rw=write --bs=1M --size=4G --numjobs=1 --direct=1
fio --name=seqread  --rw=read  --bs=1M --size=4G --numjobs=1 --direct=1

See Storage and Ports for the difference between local NVMe and network volumes — expected throughput differs substantially between the two.

If results are below expectations

Re-run the failing test at least twice to rule out transient load.
Capture the output of nvidia-smi -q alongside the benchmark output.
Contact support with the instance ID, region, and the collected outputs.

Overview

GPUs

References

Account & Billing

Verifying Instance Performance

Quick health check

GPU diagnostics (DCGM)

Compute throughput

Memory bandwidth

Multi-GPU and interconnect

Storage I/O

If results are below expectations

​Quick health check

​GPU diagnostics (DCGM)

​Compute throughput

​Memory bandwidth

​Multi-GPU and interconnect

​Storage I/O

​If results are below expectations

Quick health check

GPU diagnostics (DCGM)

Compute throughput

Memory bandwidth

Multi-GPU and interconnect

Storage I/O

If results are below expectations