Your graphics card's VRAM decides which AI models you can run, and it behaves like a wall rather than a slope. A model either fits in video memory or it does not. The moment it does not, throughput drops by roughly five to twenty times. Almost everything else is detail.
Here is what fits at each tier, checked against real model file sizes rather than rules of thumb.
The only rule you actually need
A quantised model needs roughly as much VRAM as its download size, plus one to two gigabytes for context and overhead. Ollama publishes the file size of every model it hosts, so you can check any model against your card in about ten seconds.
Sizes below are the current defaults from Ollama's own library, checked August 2026.
- Qwen3 8B 5.2 GB
- Qwen3 14B 9.3 GB
- gpt-oss 20B 14 GB
- Qwen3 30B 19 GB
- Qwen3 32B 20 GB
- gpt-oss 120B 65 GB
8GB: smaller models, and they are better than you expect
An 8GB card comfortably runs 7B and 8B models. Qwen3 8B at 5.2 GB leaves room for a reasonable context window, and on a modern card it responds faster than you can read.
This tier covers the RTX 5050, the RTX 5060 and the 8GB version of the RTX 5060 Ti. It is genuinely useful for summarising, drafting, code completion and answering questions about documents you feed it. What it will not do is run the larger reasoning models, and no amount of patience changes that.
12GB: an awkward middle
The RTX 5070 has 12GB. That clears the 14B class with room to spare, which is a real step up in quality over 8B. It does not clear gpt-oss 20B, which wants about 16GB.
If gaming is the main job and local AI is a bonus, 12GB is fine. If local AI is a specific goal, the next tier is where the useful jump happens.
16GB: the practical sweet spot
16GB is where local AI stops feeling like a compromise. It runs the 14B class comfortably and clears gpt-oss 20B, which is the model most people mean when they say a local model finally felt good enough.
On the NVIDIA side that is the RTX 5070 Ti and the RTX 5080. On the AMD side, the RX 9070 XT and the RX 9060 XT both carry 16GB, and the 9060 XT reaches this tier at a notably lower price than anything comparable from NVIDIA.
32GB: a different category
The RTX 5090 has 32GB, which runs 30B and 32B models entirely in VRAM. That is the point where a local model stops being a toy for testing and starts being something you would actually choose over a hosted one for private work.
It still will not run gpt-oss 120B. That model is 65 GB and wants an 80GB datacentre card. Nothing in a desktop tower gets there, and machines built for that job, like the NVIDIA DGX Spark, solve it with a different memory architecture rather than a bigger GPU.
What happens when it does not fit
This is the part worth understanding before you buy, because it is counterintuitive.
If a model is too large for your VRAM, Ollama and llama.cpp will not refuse. They quietly move the excess layers into system RAM and keep going. It works, and it is dramatically slower: reported penalties run from about five times to twenty times, because the GPU spends most of its time waiting for data to arrive across the PCIe bus instead of computing.
A 70B model on a 12GB card is the extreme version. It runs. It produces one to three tokens per second, which is slower than reading aloud. The practical rule is that a smaller model fully in VRAM beats a larger model spilling out of it, almost every time.
AMD or NVIDIA?
Worth answering plainly, because the old advice is out of date.
AMD used to be a poor choice for this. It no longer is. ROCm 7.2, released in March 2026, was the first version with proper RDNA 4 support and working parity across Ollama, LM Studio and llama.cpp. AMD's own documentation lists both the RX 9070 XT and the RX 9060 XT as fully supported on Windows, which needs Windows 11 22H2 or newer.
NVIDIA still holds an advantage. Independent testing puts ROCm somewhere between roughly 10 and 30 percent behind CUDA depending on the workload and runtime, and CUDA remains the default target for new tools, so NVIDIA tends to work first. There is also a narrower point in NVIDIA's favour: gpt-oss ships in a four-bit format called MXFP4 that Blackwell cards handle natively, and the RTX 50 series qualifies. On hardware without native support the model still runs, but the runtime falls back to a wider format and part of the memory advantage disappears.
If local AI is a serious goal, NVIDIA is the safer default. If you want the most VRAM per Ringgit and are willing to occasionally wait for tooling to catch up, the 16GB Radeon cards are a legitimate choice rather than a compromise.
The part most guides skip: system RAM
VRAM gets all the attention, but system RAM sets your ceiling for anything that does not fit on the GPU, and it is also the thing that has become expensive. Memory prices have roughly tripled in the past year, so adding RAM later is no longer the cheap escape hatch it used to be. Our explainer on RAM prices covers why.
The practical consequence: if running local models is a real goal rather than an experiment, say so when you spec the machine. Buying the memory up front costs less than adding it in a year, and it is the difference between a model that runs and a model that crawls.
What to install
Two tools cover almost everyone. LM Studio gives you a normal desktop application, a model browser and a chat window, and it will tell you whether a model fits your card before you download it. Ollama is command line first with a built-in library and an API, which suits you better if you intend to connect the model to something else.
Both are free, both run on Windows, and both work on NVIDIA and on the supported Radeon cards. Start with LM Studio if you are not sure.
What we would tell you
Pick your VRAM tier for the largest model you actually intend to run, not the largest one you have read about. 8GB is genuinely useful, 16GB is the point where it stops feeling limited, and 32GB is a different category of machine.
Then check the download size before you commit to anything. A model that fits is worth more than a model that impresses on paper and then spills into system RAM.
The full custom gaming PC range lists the exact GPU and VRAM in every build. If you tell us which models you want to run, over WhatsApp, you will get a spec back that fits them with the reasoning attached.
Model sizes and support status verified August 2026. This area moves quickly, so check current file sizes before buying hardware around a specific model.
