Can I Run This AI Model Locally?

Pick your GPU to see which open AI models it can run and how fast, or pick a model to see how much memory it needs and which GPUs can handle it.

Your Model

Model

Compression

Context length

gpt-oss 120B at MXFP4 needs about 62 GB

The format the model is released in.

Model weights: 61 GB
Context (KV cache): 0.3 GB
Runtime overhead: 1.0 GB
Total: 62 GB

Runs entirely on a GPU with 66 GB of VRAM or more.

Apache 2.0 · Hugging Face

GPUs that run it entirely on the GPU

Smallest VRAM first. Speeds are estimates for one GPU.

No single GPU in our database has enough VRAM. Running it locally takes several GPUs, a workstation card, or offloading most of it to system RAM, which is very slow.

How the estimate is calculated

The numbers follow how runtimes like llama.cpp, Ollama and LM Studio use memory. Here is exactly how they are produced.

Model weights

Every parameter is stored at the chosen compression level. Q4 averages just under 5 bits per weight once block scales are included; FP16 uses the full 16.

weights = parameters × bitsPerWeight / 8

Q4 4.85 · Q5 5.69 · Q8 8.5 · FP16 16 bits. gpt-oss uses its published MXFP4 checkpoint size.

Context memory (KV cache)

The model keeps keys and values for every token in the conversation. The size per token comes from each model's published architecture; sliding-window layers only keep their window.

kv = 2 × layers × kvHeads × headDim × 2 bytes × tokens

Plus about 1 GB of runtime overhead for the GPU context and compute buffers.

Does it fit?

About 95% of VRAM is usable after the desktop and driver take their share. Whatever doesn't fit can spill into system RAM, of which we assume 75% is free for the model.

total ≤ 0.95 × VRAM → On GPU
spill ≤ 0.75 × RAM → GPU + RAM

Anything larger won't run without more hardware.

Speed estimate

Generating a token means reading every active weight once, so speed is set by memory bandwidth. Mixture-of-experts models only read their active parameters, which is why they're fast for their size.

seconds = gpuPart / (0.7 × gpuBW)
        + ramPart / (0.6 × ramBW) + 1.5 ms

Single-user generation speed. Prompt processing is usually much faster.

Estimates are a guide, not a guarantee. Real memory use and speed vary with the runtime, its settings, drivers and operating system. GPU memory bandwidths are reference specifications.