GPU server infrastructure

Our GPU server infrastructure is built for work that runs for weeks, not minutes. Every server is either a bare-metal machine or a dedicated VM with a whole GPU passed through, housed in professional data centers and monitored around the clock. This page explains the hardware we deploy, how each server is balanced, how it connects to the network, and how we keep it running.

The hardware we deploy

We focus on current-generation accelerators for AI training, inference and rendering, in three formats.

8-GPU servers, bare metal

NVIDIA HGX B300, B200, H200 and H100, AMD Instinct MI355X, and RTX PRO 6000 Blackwell servers with eight GPUs. On HGX boards, the eight SXM GPUs are linked by NVLink at 1.8 TB/s per GPU on Blackwell and 900 GB/s on Hopper, per NVIDIA's published specifications. The MI355X platform connects its eight GPUs over AMD Infinity Fabric, and the RTX PRO 6000 is a PCIe Gen5 card without NVLink. See our 8-GPU servers for the full lineup.

Single-GPU servers

A dedicated virtual machine with the whole GPU passed through: B300, B200, H200, H100 SXM, RTX PRO 6000 Blackwell, L40S or GeForce RTX 5090. You get all of the GPU's memory and compute, with the vCPU, RAM and NVMe listed for each single-GPU server.

Rack-scale systems on request

For frontier-scale training, NVIDIA GB300 NVL72 racks join 72 Blackwell Ultra GPUs in a single NVLink domain, liquid-cooled, on 12- to 36-month terms. Reservations are open for Vera Rubin NVL72. Both are scoped with you directly; see rack-scale systems.

Every server ships with one of our OS images: Ubuntu 22.04 or 24.04 with NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled, ROCm images for AMD, and Debian or Rocky Linux on request.

Balanced servers, not just GPUs

A fast GPU waits on everything around it. Data loaders need CPU cores, dataset caches need RAM, and checkpoints need fast local disks. We size the rest of each server to keep its GPUs fed.

Single-GPU servers scale the same way, from 16 vCPU, 64 GB of RAM and 1 TB of NVMe with an RTX 5090 to 28 vCPU, 384 GB of RAM and 3.5 TB of NVMe with a B300. Local NVMe is fast working storage for datasets and checkpoints. It is not backed up by us unless agreed in writing, so keep copies of anything you cannot recreate.

Network and bandwidth

Every server gets a public IPv4 address and an IPv6 /64, with DDoS mitigation on its public IPs.

• Single-GPU servers: 25 Gbps on B300, B200, H200 and H100; 10 Gbps on RTX PRO 6000, L40S and RTX 5090.
• 8-GPU servers: eight 800G ConnectX-8 NICs on HGX B300, eight 400G ConnectX-7 NICs on HGX B200, H200 and H100, eight 400G RoCE NICs on MI355X, and two 100 GbE ports on RTX PRO 6000 servers.
• Private networking: available on request between your servers, so internal traffic does not have to cross public interfaces.

Bandwidth is included with no egress fees: 20 TB per month on single-GPU plans and 50 TB on 8-GPU servers, under fair use, and inbound traffic is not counted. Our bandwidth and egress guide explains how the allowance works. Planning to train across several servers? See GPU clusters.

Data center GPU servers, with the location in writing

Our data center GPU servers run in professional data centers, not on peer-to-peer hosts. The location of each server is stated in the written order confirmation you receive before paying, and that confirmation forms part of your contract. You know where your data will live before you commit, which matters if you have data-residency rules to follow or users you want to stay close to. If location matters to you, say so in your request.

How we operate our GPU server infrastructure

We monitor every server around the clock, watching GPU health, temperatures and links. Failed GPUs and components are replaced as a priority, and when a repair cannot be completed quickly, we offer an equivalent server where available.

Our service level agreement commits to 99.9% monthly availability for power, network and hardware on every server. If we fall short, you can request a credit on that server's monthly fee, applied to your next invoice or refunded if your service ends:

• Below 99.9% 10% credit
• Below 99.5% 25% credit
• Below 99.0% 50% credit
• Below 95.0% 100% credit

Disks are securely erased between customers. When your last paid month ends, your server is decommissioned and its disks are erased before the hardware is used again, so your data never meets the next tenant.