LLM GPU requirements
Qwen3 235B-A22B GPU requirements
How much GPU memory Qwen3 235B-A22B needs in BF16, FP8 and 4-bit, how its KV cache grows with context, what fine-tuning takes, and which servers run it, with monthly prices.
- Parameters
- 235B 22B active
- Architecture
- MoE 94 layers
- KV heads
- 4 × 128 dim
- Context
- 32k tokens
Cheapest servers that run it
FP8 weights · 8k context · 8 concurrent requests · about 272 GB
-
8× NVIDIA RTX PRO 6000 Blackwell In stock
8 × 96 GB GDDR7 · 8-GPU server · PCIe, no NVLink
$10,490/mo
-
8× NVIDIA HGX H100 In stock
8 × 80 GB HBM3 · 8-GPU server
$16,900/mo
-
8× AMD Instinct MI355X Limited
8 × 288 GB HBM3E · 8-GPU server
$19,900/mo
Delivery date confirmed before you pay. Missed it? Full refund.
Inference memory
How much VRAM Qwen3 235B-A22B needs
Weights take parameters × bytes per parameter. Serving adds the KV cache and about 10% runtime overhead; the totals below assume an 8k context with 8 concurrent requests and a BF16 KV cache.
| Precision | Weights | Total to serve | Cheapest server that fits |
|---|---|---|---|
| BF16 2 bytes per parameter | 470 GB | 531 GB | 8× NVIDIA RTX PRO 6000 Blackwell $10,490/mo |
| FP8 1 byte per parameter | 235 GB | 272 GB | 8× NVIDIA RTX PRO 6000 Blackwell $10,490/mo |
| 4-bit 0.5 bytes per parameter | 118 GB | 143 GB | NVIDIA B200 $4,590/mo |
Estimates with about 8% of GPU memory kept free. Quantized weights also keep some layers in higher precision, and frameworks differ in overhead; check the free memory your engine reports at startup. A model split across GPUs with tensor parallelism runs best on NVLink servers (HGX); the 8× RTX PRO 6000 server links its GPUs over PCIe.
KV cache
Qwen3 235B-A22B KV cache by context length
Each token in flight stores keys and values for every layer: 2 × 94 layers × 4 KV heads × 128 dimensions × 2 bytes = 188 KB per token in BF16. An FP8 KV cache halves these figures.
| Context per request | 1 request | 8 requests | 32 requests |
|---|---|---|---|
| 4k tokens | 0.8 GB | 6.3 GB | 25 GB |
| 32k tokens | 6.3 GB | 50 GB | 202 GB |
| 128k tokens | 25 GB | 202 GB | 807 GB |
Fine-tuning
Memory to fine-tune Qwen3 235B-A22B
Rough totals before tuning batch size and sequence length: QLoRA keeps 4-bit base weights, LoRA keeps BF16 weights frozen, and a full fine-tune with AdamW needs about 16 bytes per parameter plus activations, sharded across GPUs.
| Method | Approximate memory | Cheapest server that fits |
|---|---|---|
| QLoRA (4-bit base) | 177 GB | NVIDIA B300 $5,190/mo |
| LoRA (BF16 base) | 583 GB | 8× NVIDIA RTX PRO 6000 Blackwell $10,490/mo |
| Full fine-tune (AdamW, mixed precision) | 4,512 GB | Multi-node: GPU clusters |
Full fine-tuning assumes sharded training (FSDP or ZeRO-3) with activation checkpointing. See the fine-tuning guide for methods and settings.
About Qwen3 235B-A22B
Qwen3 235B-A22B is the flagship Mixture-of-Experts model of the Qwen3 family: 235B parameters with about 22B active per token, 94 layers and only 4 KV heads, which keeps its KV cache small for its size. Every expert must still sit in GPU memory, so the weights decide the server.
Every figure on this page comes from the model’s published architecture and the same formulas as our VRAM calculator, where you can change the context length, concurrency and KV cache precision.
Model facts
- Developer
- Alibaba Qwen
- License
- Apache 2.0
- Type
- Mixture of Experts
- Parameters
- 235Babout 22B active per token
- Layers
- 94
- Attention
- 64 query heads, 4 KV headsgrouped-query attention, head dimension 128
- Context window
- 32k tokens native, about 128k with YaRN
FAQ
Qwen3 235B-A22B GPU FAQ
The short version. Anything else, our engineers reply 24/7.
Ask an engineerHow much VRAM does Qwen3 235B-A22B need?
About 470 GB for the weights in BF16, 235 GB in FP8 and 118 GB in 4-bit, plus the KV cache: about 0.2 GB per 1,000 tokens in flight with a BF16 cache. Serving it in FP8 with an 8k context and 8 concurrent requests takes about 272 GB in total.
Can Qwen3 235B-A22B run on a single GPU?
Not at an 8k context with 8 concurrent requests: in FP8 it needs about 272 GB, more than any single GPU holds. An 8-GPU server such as the 8× NVIDIA RTX PRO 6000 Blackwell runs it with tensor parallelism.
What is the cheapest FullGPU server for Qwen3 235B-A22B?
For FP8 serving with an 8k context and 8 concurrent requests, the 8× NVIDIA RTX PRO 6000 Blackwell at $10,490 per month is the lowest-priced configuration that fits. For other contexts and concurrency, size it in the VRAM calculator or ask an engineer.
How much memory do I need to fine-tune Qwen3 235B-A22B?
Roughly 177 GB with QLoRA, 583 GB with LoRA on BF16 weights, and 4,512 GB for a full fine-tune with AdamW in mixed precision, before tuning batch size and sequence length. Full fine-tunes of large models are sharded across GPUs with FSDP or ZeRO-3.
Talk to an engineer
Run Qwen3 235B-A22B on your own GPUs.
Tell us what you need. A GPU engineer replies with availability, a confirmed delivery date and a payment link.
- Delivery date confirmed before you pay
- Cancel any month
- Card or crypto, same price