Can I Run This AI Model Locally?
Pick your GPU to see which open AI models it can run and how fast, or pick a model to see how much memory it needs and which GPUs can handle it.
How the estimate is calculated
The numbers follow how runtimes like llama.cpp, Ollama and LM Studio use memory. Here is exactly how they are produced.
Model weights
Every parameter is stored at the chosen compression level. Q4 averages just under 5 bits per weight once block scales are included; FP16 uses the full 16.
weights = parameters × bitsPerWeight / 8Q4 4.85 · Q5 5.69 · Q8 8.5 · FP16 16 bits. gpt-oss uses its published MXFP4 checkpoint size.
Context memory (KV cache)
The model keeps keys and values for every token in the conversation. The size per token comes from each model's published architecture; sliding-window layers only keep their window.
kv = 2 × layers × kvHeads × headDim × 2 bytes × tokensPlus about 1 GB of runtime overhead for the GPU context and compute buffers.
Does it fit?
About 95% of VRAM is usable after the desktop and driver take their share. Whatever doesn't fit can spill into system RAM, of which we assume 75% is free for the model.
total ≤ 0.95 × VRAM → On GPU
spill ≤ 0.75 × RAM → GPU + RAMAnything larger won't run without more hardware.
Speed estimate
Generating a token means reading every active weight once, so speed is set by memory bandwidth. Mixture-of-experts models only read their active parameters, which is why they're fast for their size.
seconds = gpuPart / (0.7 × gpuBW)
+ ramPart / (0.6 × ramBW) + 1.5 msSingle-user generation speed. Prompt processing is usually much faster.
Estimates are a guide, not a guarantee. Real memory use and speed vary with the runtime, its settings, drivers and operating system. GPU memory bandwidths are reference specifications.