DeepSeek R1 hardware requirements: GPUs and memory for R1 and V3

DeepSeek R1 hardware requirements come down to one number: about 671 GB of GPU memory for the weights in FP8, the precision DeepSeek released them in, plus room for the KV cache. That puts the full model on an 8-GPU server, not a single card. DeepSeek-V3 has the same architecture and size, so its GPU requirements are the same. The distilled R1 models are a different story: they are ordinary dense models from 1.5B to 70B parameters, and each runs on one GPU. This guide covers both, with the arithmetic.

DeepSeek R1 hardware requirements at a glance

• Architecture Mixture-of-Experts, 671B parameters in total, about 37B activated per token
• Released precision FP8 (E4M3) weights with block-wise scales
• Weights in GPU memory about 671 GB in FP8, about 1.3 TB in BF16
• KV cache small per token, thanks to Multi-head Latent Attention
• Smallest comfortable server 8× H200, 1,128 GB
• Room for BF16 or very long contexts 8× B300 or 8× MI355X, 2,304 GB
• Not enough for FP8 weights 8× H100, 640 GB
• Distilled R1 models 1.5B to 70B parameters, one GPU each

DeepSeek-R1 was trained from the DeepSeek-V3 base model, so the memory math below applies to both.

Why 37B active parameters still need 671 GB

In a Mixture-of-Experts model, most layers replace one large feed-forward block with many smaller experts. DeepSeek-V3 and R1 use 256 routed experts plus one shared expert in each MoE layer, and a router selects 8 of the routed experts for each token. Different tokens pick different experts, so all of them must sit in GPU memory, ready for whichever token needs them next.

The 37B activated parameters set the compute and the bytes read per generated token, which is why decode is relatively cheap for a model of this size. They do not set the memory you need. Some runtimes keep experts in CPU memory instead, but then every token waits on much slower system memory or CPU compute: workable for experiments, a poor fit for production serving.

Memory for the weights: FP8, BF16 and 4-bit

DeepSeek trained V3 with an FP8 mixed-precision framework and published the weights in FP8, with a scale for each 128 × 128 block of the quantized weight matrices. At one byte per parameter, the weights take about 671 GB. The published checkpoint also includes a 14B-parameter multi-token prediction module, 685B parameters in all, so plan for roughly 700 GB of disk.

Two points are easy to miss. First, because the official weights are FP8, a BF16 copy is an upcast: it holds essentially the same information in twice the memory. It is useful on software paths without FP8 support and in some fine-tuning workflows, not for better answers. Second, going below FP8 does change the model. 4-bit versions bring the full model within reach of 8× H100 or 8× RTX PRO 6000, but measure the quality on your own tasks before relying on one. Our explainer on how FP8 and 4-bit formats work covers the trade-offs.

KV cache: why Multi-head Latent Attention keeps it small

Standard multi-head attention caches a key and a value vector for every attention head, in every layer, for every token. Multi-head Latent Attention (MLA) caches a compressed latent instead: per layer and per token, 512 latent values plus a 64-value positional key, 576 values in all, shared by all 128 attention heads.

The arithmetic, with 61 layers and a BF16 cache: 61 × 576 × 2 bytes = 70,272 bytes, about 70 KB per token.

• One 32K-token sequence: about 2.3 GB.
• One 128K-token sequence: about 9.2 GB.

For comparison, the 70B distilled model, built on a Llama 70B base with 80 layers and 8 KV heads of 128 dimensions, needs about 0.33 MB per token in BF16: roughly 4.7 times more per token than the full R1, for a model about a tenth the size. How the cache is spread across GPUs depends on the engine and its parallelism mode, so rely on the KV capacity your engine reports at startup rather than on hand math.

Reasoning makes the cache matter anyway. R1 writes out its chain of thought before it answers, and traces of thousands of tokens are common, so plan for long outputs as well as long prompts.

DeepSeek V3 GPU requirements: which 8-GPU servers fit

Split evenly across eight GPUs, the FP8 weights take about 84 GB per GPU and the BF16 weights about 168 GB. Whatever is left on each GPU goes to the KV cache, activations and the runtime.

The free-memory figures are upper bounds computed from nominal capacity; drivers, the runtime and activation buffers take a share. Even so, the margins are large. Suppose an engine kept a full copy of the latent cache on every H200 and had 45 GB per GPU left for it: at about 70 KB per token, that is still around 640,000 tokens, for example 19 concurrent 32K-token conversations. Engines that split the cache across GPUs fit more.

If you already run 8× H100 servers, two of them linked over private networking give 1,280 GB, enough for the FP8 weights with pipeline parallelism across the pair. It works, but a single 8× H200 is simpler to operate. For the AMD option, see our notes on the AMD MI355X for AI; on the NVIDIA side, the NVIDIA H200 page lists the HGX H200 configuration and its stock.

Running the R1 distilled models on one GPU

Alongside R1, DeepSeek released six dense models fine-tuned on reasoning data generated by R1, built on Qwen2.5 and Llama 3 base models. They use conventional grouped-query attention, so the usual memory math applies.

Size the KV cache generously, because these models also write long reasoning traces. A distilled 32B on an RTX PRO 6000 at $1,390 per month, or the 70B on an H200 at $3,090, is an inexpensive way to put R1-style reasoning into production and find out whether your users need the full model. Our guide to the best GPU for LLM inference applies the same sizing method to other models.

Serving DeepSeek R1 with vLLM or SGLang

Both vLLM and SGLang support this model family, including MLA and the FP8 weights, and both run on NVIDIA GPUs and, through ROCm, on AMD Instinct GPUs. A minimal start on one 8-GPU server:

# vLLM: tensor parallelism across all eight GPUs, context capped at 32K tokens
vllm serve deepseek-ai/DeepSeek-R1 --tensor-parallel-size 8 --max-model-len 32768

# SGLang: the same server
python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1 --tp 8 --trust-remote-code

# A distilled model on a single GPU
vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --max-model-len 32768

A few practical notes:

• Disk and download. Our HGX servers carry 8 × 3.84 TB of local NVMe, so the roughly 700 GB checkpoint fits with room for converted copies, and inbound traffic does not count toward the included bandwidth.
• Cap the context. Set the maximum model length to what your application needs rather than the model's full window; the engine checks it against the cache it can allocate.
• Read the startup log. The engine reports its KV cache capacity after loading the weights, and that is the number to plan concurrency with.
• Handle the reasoning. R1 writes its reasoning inside <think> tags before the final answer. Budget output length for it, and decide whether your application shows it or strips it.

For a full production setup, see our guide to self-hosting LLM inference with vLLM.

Which configuration to choose

• Production serving of R1 or V3 in FP8 at moderate concurrency: 8× H200.
• High concurrency or long reasoning traces at scale: 8× B200.
• BF16 weights or very long contexts: 8× B300, or 8× MI355X at about half the B300's monthly price if your stack runs on ROCm.
• Reasoning on a budget: a distilled model on one GPU, such as the 32B on an RTX PRO 6000 or the 70B in FP8 on an H200.

All of our 8-GPU servers are bare-metal machines rented by the month, with the delivery date confirmed in writing before you pay. To check a specific model and context length, our VRAM calculator does the math.

Frequently asked questions

How much VRAM does DeepSeek R1 need?

About 671 GB for the weights in FP8, the precision DeepSeek released, plus KV cache and runtime overhead, or about 1.3 TB in BF16. In practice that means a full 8-GPU server such as 8× H200 with 1,128 GB.

Can DeepSeek R1 run on 8× H100?

Not with the FP8 weights: 8× H100 has 640 GB, less than the roughly 671 GB the weights need. You would need 4-bit quantization, with an accuracy cost to measure, or two servers.

Does DeepSeek V3 have the same GPU requirements as R1?

Yes. Both are 671B-parameter Mixture-of-Experts models with about 37B parameters activated per token and the same architecture, so the memory math and the server choices are identical.

Which GPU do I need for the DeepSeek R1 distilled models?

The 1.5B, 7B and 8B models run on an RTX 5090, the 14B on an L40S, the 32B on an RTX PRO 6000 or H100, and the 70B on an H200 in FP8 or a B200 in BF16.

Can I run DeepSeek R1 on AMD GPUs?

Yes. An 8× MI355X server has 2,304 GB of HBM3E, enough for the FP8 or BF16 weights, and both vLLM and SGLang support ROCm. Validate your engine version on the model before going to production.