Can I Run This AI Model Locally?

Pick your GPU to see which open AI models it can run and how fast, or pick a model to see how much memory it needs and which GPUs can handle it.

Your Hardware

GPU

System RAM

RAM type

Context length

10 of 18 models run entirely on the RTX 4070 Ti SUPER

16 GB VRAM, 32 GB system RAM. Each model is shown at the best compression level that fits.

Model	Runs	Memory	Speed
Llama 3.2 3B FP16	On GPU	7.9 GB	~66 tok/s
Gemma 3 4B FP16	On GPU	9.3 GB	~51 tok/s
Mistral 7B v0.3 Q8	On GPU	9.2 GB	~56 tok/s
Llama 3.1 8B Q8	On GPU	9.9 GB	~51 tok/s
Qwen3 8B Q8	On GPU	10 GB	~50 tok/s
Gemma 3 12B Q8	On GPU	14 GB	~34 tok/s
Mistral Nemo 12B Q8	On GPU	14 GB	~34 tok/s
Phi-4 14B Q5	On GPU	12 GB	~42 tok/s
Qwen3 14B Q5	On GPU	12 GB	~42 tok/s
gpt-oss 20B MXFP4 · MoE, 3.6B active	On GPU	14 GB	~153 tok/s
Mistral Small 3.1 24B Q4	GPU + RAM	16 GB	~23 tok/s
Gemma 3 27B Q4	GPU + RAM	18 GB	~13 tok/s
Qwen3 30B-A3B Q4 · MoE, 3.3B active	GPU + RAM	19 GB	~77 tok/s
Qwen3 32B Q4	GPU + RAM	21 GB	~6.4 tok/s
Llama 3.3 70B Q4	Won't run	43 GB	—
gpt-oss 120B MXFP4 · MoE, 5.1B active	Won't run	62 GB	—
Qwen3 235B-A22B Q4 · MoE, 22B active	Won't run	135 GB	—
DeepSeek R1 671B Q4 · MoE, 37B active	Won't run	380 GB	—

How the estimate is calculated

The numbers follow how runtimes like llama.cpp, Ollama and LM Studio use memory. Here is exactly how they are produced.

Model weights

Every parameter is stored at the chosen compression level. Q4 averages just under 5 bits per weight once block scales are included; FP16 uses the full 16.

weights = parameters × bitsPerWeight / 8

Q4 4.85 · Q5 5.69 · Q8 8.5 · FP16 16 bits. gpt-oss uses its published MXFP4 checkpoint size.

Context memory (KV cache)

The model keeps keys and values for every token in the conversation. The size per token comes from each model's published architecture; sliding-window layers only keep their window.

kv = 2 × layers × kvHeads × headDim × 2 bytes × tokens

Plus about 1 GB of runtime overhead for the GPU context and compute buffers.

Does it fit?

About 95% of VRAM is usable after the desktop and driver take their share. Whatever doesn't fit can spill into system RAM, of which we assume 75% is free for the model.

total ≤ 0.95 × VRAM → On GPU
spill ≤ 0.75 × RAM → GPU + RAM

Anything larger won't run without more hardware.

Speed estimate

Generating a token means reading every active weight once, so speed is set by memory bandwidth. Mixture-of-experts models only read their active parameters, which is why they're fast for their size.

seconds = gpuPart / (0.7 × gpuBW)
        + ramPart / (0.6 × ramBW) + 1.5 ms

Single-user generation speed. Prompt processing is usually much faster.

Estimates are a guide, not a guarantee. Real memory use and speed vary with the runtime, its settings, drivers and operating system. GPU memory bandwidths are reference specifications.