GPU comparison
MI355X vs H200: double the memory or the proven stack?
The MI355X gives each GPU 288 GB of HBM3E against 141 GB on the H200, and adds FP6 and FP4 formats that Hopper lacks. The H200 answers with the mature Hopper and CUDA stack and a single-GPU plan at $3,090 a month.
Choose the MI355X if…
from $19,900/mo
- One model instance needs more than 141 GB per GPU.
- You want 2,304 GB of HBM3E in one server for $19,900 a month.
- You want to serve FP4 or FP6 models with native low-precision math.
- Your stack runs on PyTorch or vLLM, both of which support ROCm.
Choose the H200 if…
from $3,090/mo
- One GPU is enough: a single H200 is $3,090 a month.
- Your code relies on CUDA-only libraries or custom CUDA kernels.
- Your training and serving recipes are tuned for Hopper and you want no porting work.
Bottom lineIf your software runs on ROCm, the MI355X server gives you twice the memory and newer precision formats for $4,000 a month less than an HGX H200; for CUDA-only stacks or a single GPU, choose the H200.
Side by side
MI355X vs H200 specifications
Vendor-published figures per GPU. The larger value is highlighted, with the difference shown next to it.
| Specification | AMD Instinct MI355X | NVIDIA H200 |
|---|---|---|
| Architecture | CDNA 4AMD | HopperNVIDIA |
| GPU memory | 288 GB 2.0×HBM3E | 141 GBHBM3e |
| Memory bandwidth | 8 TB/s +67%per GPU | 4.8 TB/sper GPU |
| FP4 | 10,100 TFLOPSDense | —Not supported |
| FP8 | 5,000 TFLOPS 2.5×10,100 TFLOPS with sparsity | 1,979 TFLOPS3,958 TFLOPS with sparsity |
| BF16 / FP16 | 2,500 TFLOPS 2.5×5,000 TFLOPS with sparsity | 989 TFLOPS1,979 TFLOPS with sparsity |
| GPU interconnect | Infinity Fabricfull mesh between the 8 GPUs | NVLink 4900 GB/s per GPU |
| Form factor | OAM module · liquid-cooled | SXM module · HGX H200 |
| Max power | 1,400 W | 700 W |
Compute figures are dense; sparsity figures shown where vendors publish them. Prices in USD per month, excluding VAT; per-GPU-hour equivalents assume a 730-hour month. GPUs needed: weights in FP8 plus about 20% for the KV cache and runtime, within one server.
| Price | MI355X | H200 |
|---|---|---|
| 1 GPU, per month | Not offered | $3,090 |
| 8-GPU server, per month | $19,900 Lower | $23,900 |
| ≈ Per GPU-hour | $3.41 Lower | $4.23 |
| Per GB of GPU memory, per month | $8.64 Lower | $21.91 |
| Order |
| Model size | MI355X | H200 |
|---|---|---|
| 8B | 1 GPU | 1 GPU |
| 14B | 1 GPU | 1 GPU |
| 32B | 1 GPU | 1 GPU |
| 70B | 1 GPU | 1 GPU |
| 120B | 1 GPU | 1 GPUtight |
| 235B | 1 GPU | 2 GPUs |
| 405B | 2 GPUs | 4 GPUs |
The MI355X vs H200 decision is a trade between hardware and habit. AMD's Instinct MI355X has 288 GB of HBM3E per GPU, about twice the H200's 141 GB, with 8 TB/s of memory bandwidth against 4.8 TB/s and hardware support for FP6 and FP4. NVIDIA's H200 keeps the Hopper architecture of the H100 but moves to larger, faster HBM3e, and runs the CUDA stack most AI code is written for. We rent both, so this page sticks to what you can check before ordering: memory, formats, interconnect, price and porting effort.
MI355X vs H200 at a glance
On paper, the MI355X leads on every hardware line. AMD lists 5.0 PFLOPS of dense FP8 for the MI355X, about 2.5 times the 1,979 TFLOPS NVIDIA lists for the H200. It also has about twice the memory and about 1.7 times the bandwidth. The H200's strengths are elsewhere: CUDA compatibility, years of production use of Hopper across the industry, and a single-GPU plan.
Price favors the MI355X as well. Our 8× MI355X server is $19,900 a month and our HGX H200 server is $23,900: $4,000 less for twice the memory. If your software runs on ROCm, the MI355X is the stronger server. If it does not, its hardware lead does you no good.
Memory: 288 GB vs 141 GB per GPU
Capacity decides how many GPUs a model needs and how much room is left for KV cache. Weights take roughly 2 bytes per parameter in BF16, 1 in FP8 and 0.5 in FP4; plan 10–30% more for KV cache, activations and overhead. The table counts weights only.
| Model and precision | Weights, roughly | H200 GPUs needed | MI355X GPUs needed |
|---|---|---|---|
| 70B dense, FP8 | 70 GB | 1 | 1 |
| 70B dense, BF16 | 140 GB | 1 with no room to spare, 2 in practice | 1 |
| 405B dense, FP8 | 405 GB | 3 | 2 |
| 405B dense, BF16 | 810 GB | 6 | 3 |
| 671B mixture-of-experts, FP8 | 671 GB | 5 | 3 |
| 671B mixture-of-experts, BF16 | 1,342 GB | 10, more than one 8-GPU server | 5 |
The last two lines show the clearest split. DeepSeek-V3, a 671B mixture-of-experts model, was released with FP8 weights. Both 8-GPU servers hold them, but the MI355X server leaves about 1.6 TB for KV cache against about 450 GB on the HGX H200. In BF16, only the MI355X server holds the model at all, with roughly 960 GB to spare. Our DeepSeek hardware guide works through this case in detail.
Long contexts show the same gap on one GPU. A 70B-class model with grouped-query attention (80 layers and 8 KV heads, as in Llama 3 70B) needs about 0.16 MB of KV cache per token in FP8, so one 128k-token context takes about 21 GB. With FP8 weights loaded, an H200 has roughly 70 GB left: about three full-length contexts at once. An MI355X has roughly 218 GB left: about ten, before overhead.
Compute and precision formats
AMD lists the MI355X at 2.5 PFLOPS of dense BF16 and 5.0 PFLOPS of dense FP8. NVIDIA lists the H200 at 989 and 1,979 TFLOPS for the same formats, the same figures as the H100, since both use the same Hopper compute. Roughly 2.5 times the peak math is a generational gap, not a tuning difference. Peak figures are still not delivered throughput: your kernels and libraries decide how much of it you see.
The wider gap is at low precision. Hopper's lowest-precision floating-point tensor format is FP8. AMD's CDNA 4 adds MXFP6 and MXFP4, the microscaling formats standardized through the Open Compute Project, and AMD lists 10.1 PFLOPS for MXFP4. You can still serve 4-bit weights on an H200 with weight-only quantization such as AWQ or GPTQ, but the matrix math runs at higher precision: memory use drops, compute does not rise. On the MI355X, 4-bit and 6-bit models can run their math natively where your framework supports it. Our overview of the AMD MI355X for AI goes further.
Memory bandwidth and interconnect
Each generated token requires reading the active weights and the KV cache from memory, so bandwidth often sets the pace of interactive serving at small batch sizes. AMD lists 8 TB/s for the MI355X and NVIDIA lists 4.8 TB/s for the H200: on a model that fits both, the MI355X streams that data at about 1.7 times the rate.
Inside the server, the HGX H200 connects every GPU through NVLink switches with 900 GB/s of NVLink 4 bandwidth each, so any pair communicates at full NVLink speed. The MI355X uses a direct Infinity Fabric mesh: AMD lists seven links of 153.6 GB/s per GPU, one to each peer, about 1.08 TB/s in total. All-reduce and other collectives spread across every link; a transfer between two specific GPUs uses the single link between them. For training with heavy tensor parallelism, profile both. For inference, the MI355X's capacity often means splitting a model across fewer GPUs, which cuts communication at the source.
Between servers, both have eight 400G network ports: ConnectX-7 on the HGX H200 and RoCE on the MI355X. Private networking between your servers is available on request.
Monthly cost: $19,900 vs $23,900
Our 8× MI355X server works out to about $3.41 per GPU-hour and our HGX H200 to about $4.09. Per gigabyte of GPU memory, that is about $8.64 a month against $21.19: roughly 2.5 times as much memory per dollar on the MI355X.
The H200 wins on granularity. A single H200 costs $3,090 a month and holds a 70B model in FP8 with room for KV cache. If one GPU covers your workload, there is no MI355X plan to compare it with, because we rent the MI355X only as a full 8-GPU server. Two or three single H200s also stay under $10,000 a month, though as separate virtual machines they suit independent jobs rather than one model split across GPUs.
MI355X stock is limited, and we confirm every delivery date in writing before you pay. Pricing for 6- to 12-month terms or several servers is available on request. Model your own totals in the GPU cost calculator.
Software: the Hopper stack vs ROCm
The H200 runs the same software as the H100: same CUDA builds, same kernels, same container images. If your training or serving stack works on Hopper today, it works on an H200 with more memory and no porting. When a deadline is close, that predictability has real value.
The MI355X runs ROCm. PyTorch and vLLM support it, and our ROCm images give you a ready base. The friction sits in custom CUDA kernels, which need porting to HIP, and in NVIDIA-only tools such as TensorRT-LLM, which have no AMD version. Audit your dependencies, then run your real workload for a month: the minimum term is one month and you can cancel any month with no fee. Our ROCm documentation covers the setup.
How to choose
- One GPU, 70B in FP8 or smaller: a single H200 at $3,090 a month.
- Eight GPUs, ROCm-compatible stack: the MI355X server at $19,900, for more memory, more bandwidth and native FP4.
- Eight GPUs, CUDA-only dependencies: the HGX H200 at $23,900, or a Blackwell server if you also need FP4 (see H200 vs B200).
- Unsure whether your stack ports: test on the MI355X for one month before committing to a longer term.
Not sure which one fits?
Tell us your model, precision and traffic. A GPU engineer sizes it and replies with availability and a delivery date.
Ask an engineer Try the VRAM calculatorIs the MI355X better than the H200?
On hardware, yes: twice the memory per GPU, about 1.7 times the memory bandwidth and roughly 2.5 times the vendor-published FP8 and BF16 peak. The H200 is the safer choice when your code depends on CUDA-only components, because it runs the Hopper stack unchanged.
Can a 70B model run on one H200?
Yes in FP8, where the weights take about 70 GB and leave about 70 GB for KV cache. In BF16 the weights alone take about 140 GB, so you need two H200s or a GPU with more memory, such as the MI355X or the B200.
Does the H200 support FP4?
Not natively, since Hopper's lowest-precision floating-point tensor format is FP8. You can store weights in 4-bit with weight-only quantization, but the math runs at higher precision, while the MI355X supports MXFP4 and MXFP6 in hardware.
Do I need to change my code to move from the H200 to the MI355X?
Usually not for PyTorch or vLLM code, since both support ROCm and PyTorch keeps its torch.cuda device interface on AMD GPUs. Custom CUDA kernels need porting to HIP, and NVIDIA-only tools such as TensorRT-LLM need replacing.
Which is better for DeepSeek-class models?
A 671B mixture-of-experts model takes about 671 GB in FP8. Both 8-GPU servers hold it, but the MI355X server leaves about 1.6 TB for KV cache against about 450 GB on the HGX H200, and only the MI355X server holds it in BF16.
Talk to an engineer
Get your GPU server this month.
Tell us what you need. A GPU engineer replies with availability, a confirmed delivery date and a payment link.
- Delivery date confirmed before you pay
- Cancel any month
- Card or crypto, same price