Can I run Qwen3 8B locally?
Yes. At Q4 with 8K context, Qwen3 8B needs about 6.7 GB, so it runs entirely on any GPU with 8 GB of VRAM or more, starting with the NVIDIA GeForce RTX 5060 Ti 8GB.
Memory needed by compression and context length
| Compression | 4K context | 8K context | 32K context |
|---|---|---|---|
Q4 The most common download. Small quality loss. | 6.2 GB | 6.7 GB | 10 GB |
Q5 Very close to full quality. | 7.0 GB | 7.6 GB | 11 GB |
Q8 Practically indistinguishable from FP16 at about half the size. | 9.7 GB | 10 GB | 14 GB |
FP16 Uncompressed. Reference quality, largest size. | 17 GB | 17 GB | 21 GB |
Total memory: model weights, context (KV cache) and about 1 GB of runtime overhead. To run entirely on the GPU, VRAM must be about 5% larger than this.
GPUs that run Qwen3 8B entirely on the GPU
- Amazon
NVIDIA GeForce RTX 5060 Ti 8GB
8 GB · ~58 tok/s at Q4
- Amazon
NVIDIA GeForce RTX 5060
8 GB · ~58 tok/s at Q4
- Amazon
AMD Radeon RX 9060 XT 8GB
8 GB · ~42 tok/s at Q4
- Amazon
NVIDIA GeForce RTX 4060
8 GB · ~36 tok/s at Q4
- Amazon
AMD Radeon RX 6650 XT
8 GB · ~37 tok/s at Q4
- Amazon
AMD Radeon RX 5700 XT 50th Anniversary
8 GB · ~58 tok/s at Q4
- Amazon
NVIDIA GeForce RTX 2060 SUPER
8 GB · ~58 tok/s at Q4
- Amazon
AMD Radeon RX 7600
8 GB · ~38 tok/s at Q4
- Amazon
AMD Radeon RX 6600 XT
8 GB · ~34 tok/s at Q4
- Amazon
NVIDIA GeForce GTX 1080
8 GB · ~42 tok/s at Q4
Check it with your hardware
Change the compression, context length and system RAM in the calculator.
Open calculator