Rent NVIDIA H200 servers: a single GPU or an HGX H200

The main reason to rent NVIDIA H200 capacity is memory: 141 GB of HBM3e per GPU, enough to serve a 70B model in FP8 on a single card with room to spare. We offer the H200 in stock as a single GPU in a dedicated virtual machine at $3,090 per month, our most popular single-GPU plan, and as a bare-metal HGX H200 server with eight GPUs at $23,900 per month.

What the H200 adds to the H100

The H200 uses the same Hopper GPU design as the H100. NVIDIA lists identical compute for both, 1,979 TFLOPS of dense FP8 and 989 TFLOPS of dense BF16, the same NVLink 4 at 900 GB/s per GPU and the same 700 W SXM form factor. The difference is memory: 141 GB of HBM3e instead of 80 GB of HBM3 (76% more), at 4.8 TB/s instead of 3.35 TB/s (43% more).

That matters most for inference. To generate each token, the GPU reads the model weights from memory, so bandwidth sets the ceiling on decode speed, and every gigabyte not taken by weights can hold KV cache for more concurrent requests or longer contexts. For compute-bound work, such as training a model that already fits comfortably in 80 GB, the gap narrows and the H100 may be the better value. Our H100 vs H200 comparison lays out both cases.

What fits on one H200, and on eight

Weights take roughly 2 bytes per parameter in BF16 and 1 in FP8; add KV cache and 10–30% headroom.

• 70B in FP8 on one H200: about 70 GB of weights, leaving roughly 50–60 GB for KV cache after headroom. That headroom is what makes a single H200 practical for a production 70B endpoint.
• Long contexts: a Llama-style 70B model with grouped-query attention (80 layers, 8 KV heads of dimension 128) needs about 0.33 MB of KV cache per token in BF16. One 128k-token context is therefore about 43 GB, which fits next to the FP8 weights on an H200 but not on an 80 GB GPU.
• 70B in BF16: about 140 GB, too tight for one H200. Use two GPUs of the HGX H200 with tensor parallelism, or a 180 GB B200.
• HGX H200, 1.1 TB in total: a 405B-class model in FP8 (about 405 GB) leaves roughly 700 GB for KV cache, and a 671B mixture-of-experts model in FP8 (about 671 GB) still fits with around 450 GB to spare.

For fine-tuning, QLoRA on a 70B model (about 35 GB of 4-bit weights) runs comfortably on one H200, while LoRA on 70B in BF16 does not fit one GPU because the frozen weights alone take about 140 GB. Full fine-tuning with Adam needs roughly 16–20 bytes per parameter before activations: about 512–640 GB for a 32B model, which the HGX H200 handles, but 1.1–1.4 TB for a 70B model, which exceeds its HBM3e unless you offload optimizer state to the 2 TB of system RAM. Try other sizes in our VRAM calculator.

Single H200 or an HGX H200 server?

The single H200 comes with 24 vCPUs, 240 GB of RAM, 2 TB of NVMe and a 25 Gbps port. It fits teams that serve one model per GPU and scale by adding replicas, or that fine-tune with LoRA and QLoRA.

The HGX H200 links eight GPUs in one NVLink domain at 900 GB/s per GPU, so you can split a model with tensor parallelism or shard training state across all eight without touching the network. The server has two Xeon 8480+ processors (112 cores in total), 2 TB of DDR5, eight 3.84 TB NVMe drives and eight 400G ConnectX-7 adapters. At eight GPUs it also costs less than single plans: $23,900 per month, against $24,720 for eight single H200s.

Software: a mature Hopper stack

Hopper GPUs have been widely deployed since 2023, so FP8 support in vLLM, SGLang and TensorRT-LLM is well established. Your server arrives with root access over SSH, Ubuntu 22.04 or 24.04, and NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled; Debian or Rocky Linux are available on request. Serving a 70B model from an FP8 checkpoint, with an FP8 KV cache, on one H200 looks like this:

vllm serve <your-70b-fp8-checkpoint> --kv-cache-dtype fp8 \
  --max-model-len 131072 --host 127.0.0.1 --port 8000

Storing the KV cache in FP8 halves its size, which roughly doubles the context or concurrency that fits in the same memory. Our guide to self-hosting LLM inference with vLLM covers production settings, and choosing a GPU for LLM inference compares the options.

H200 price per month, explained

• Single H200 $3,090 per month, about $4.23 per GPU-hour
• HGX H200, 8 GPUs $23,900 per month, about $4.09 per GPU-hour

The hourly figure is the monthly price divided by the number of GPUs and by 730 hours, so it holds only if the GPU runs all month. A 24/7 inference endpoint does, and there a flat $3,090 replaces a meter. An endpoint that only runs during business hours, about 220 hours a month, would cost about $14.05 per used hour on a monthly plan, and hourly billing is then likely the better choice. The general rule: divide our monthly price by the hourly rate you would pay elsewhere, and monthly wins above that number of hours. Our monthly vs hourly comparison shows the math in detail.

Prices have also been moving. FullGPU's October 2026 survey of public price lists found the median on-demand H200 price per GPU-hour up roughly 27% over the past year. A monthly plan fixes your price for each paid month. It includes outbound traffic with no egress fees (20 TB per month for the single GPU, 50 TB for the 8-GPU server, under fair use), a public IPv4 address and an IPv6 /64, DDoS mitigation, 24/7 support and the 99.9% availability SLA. There is no setup fee.

How to rent NVIDIA H200 servers

Request the plan you want on the site. We confirm in writing the configuration, price, location and delivery date before you pay anything. You then pay the first month by card or in crypto, at an exchange rate locked at checkout, and receive access details by email. If the confirmed delivery date slips, you can cancel and get a full refund. Plans renew monthly with an email reminder, and you can cancel any month without a fee. For several servers or a 6–12 month term, ask us for volume pricing.