Ollama Coding Model Setup, and Where It Falls Short

Thirty minutes. That’s how long a local coding model took on my laptop to write a hello world program in Go, a program whose entire job is to print one line.

I was running Qwen Coder through Ollama, the free tool that downloads open-weights models and runs them on your own machine. The setup worked. My laptop just didn’t have the resources to run it at any useful speed.

That experience shapes this guide. Setting up an Ollama coding model is easy on Windows, macOS, and Linux, and connecting it to VS Code takes one config file. How well it runs depends almost entirely on your hardware. And even on good hardware, it won’t replace a frontier coding agent for work that spans many files. For autocomplete and for code you can’t send to the cloud, it earns its place.

I compared local and cloud models in general in my local AI models post. This is the hands-on version, for code. Steps are checked against Ollama 0.40.14, the current release as of November 2026.

What Your Machine Needs

One rule matters more than any other: the model file has to fit in your graphics card’s memory, called VRAM. Apple Silicon Macs share one pool of memory between the processor and the graphics chip, so there the model has to fit in that pool with room left for everything else.

If it doesn’t fit, Ollama splits the model between the GPU and the much slower CPU. Speed falls off a cliff.

Ollama’s own minimums:

Windows: Windows 10 22H2 or newer. Nvidia cards need driver 551.61 or newer. AMD cards work through ROCm v7 or Vulkan drivers.
macOS: Sonoma (version 14) or newer. Apple M-series chips get GPU acceleration. Intel Macs run on the CPU only.
Linux: Nvidia cards need compute capability 5.0 or newer and driver 550 or newer. AMD cards need ROCm v7.

Disk space adds up too. Models run from under 1 GB to tens of gigabytes each.

Pick an Ollama Coding Model That Fits

Qwen’s coder models are the usual starting point. Here’s what the Ollama library lists for the sizes that make sense at home:

Model	Download size	Max context	What I’d use it for
`qwen2.5-coder:1.5b`	986 MB	32K	Autocomplete on modest hardware
`qwen2.5-coder:7b`	4.7 GB	32K	Chat and quick edits
`qwen2.5-coder:14b`	9.0 GB	32K	Better answers, if you have the memory
`qwen3-coder:30b`	19 GB	256K	Agent-style work on high-end hardware

The download size is the floor, not the total. The model also needs memory for its context window, which I’ll get to below, so leave headroom.

qwen3-coder:30b is a mixture-of-experts model: 30 billion parameters, but only 3.3 billion active for each token it produces. That makes it faster than its size suggests. It still needs the full 19 GB loaded.

One warning. Both families show their last update on Ollama as a year ago, and the library moves fast. The qwen2.5-coder sizes are still a sensible default, but check ollama.com/library for something newer before you commit to a big download.

Installing Ollama on Windows, macOS, and Linux

Windows

Download OllamaSetup.exe from ollama.com/download and run it. It installs into your user folder and doesn’t need administrator rights.
Ollama starts in the background. The ollama command works in Command Prompt, PowerShell, or Windows Terminal.
Short on space in your user folder? Set an environment variable called OLLAMA_MODELS to another folder, then quit and relaunch Ollama from the Start menu.

macOS

Download the .dmg from ollama.com/download and drag Ollama into Applications.
On first launch, it offers to link the ollama command into /usr/local/bin. Accept, so it works in Terminal.

Linux

Run the official install script:

curl -fsSL https://ollama.com/install.sh | sh

It sets Ollama up as a systemd service, so it starts with your machine. Piping a script from the internet into your shell is a habit worth questioning, so open the URL and read it first if that bothers you.

First run, any OS

ollama pull qwen2.5-coder:7b
ollama run qwen2.5-coder:7b

Ask it for a small function. Type /bye to leave the chat.

Check That It’s Actually Using Your GPU

Illustration of an Ollama coding model overflowing GPU memory and spilling onto the slower CPU

While the model is loaded, open a second terminal and run:

ollama ps

Look at two columns. PROCESSOR should say 100% GPU. If it shows a CPU share, the model didn’t fit and part of it is running on the processor. CONTEXT shows how much context Ollama gave the model.

If your first answer arrives as slowly as my hello world did, this is the first thing to check.

Pro Tip: Run ollama ps after every change you make in this guide: a bigger model, a longer context, a second model for autocomplete. The moment a CPU share appears, step back one notch.

Raise the Context Length Before You Code

The context window is how much text the model can see at once: your instructions, the code you share, and its own answer. It’s measured in tokens, chunks of text a bit smaller than a word.

Here’s the catch most guides skip. Ollama sets the default by how much VRAM you have:

Under 24 GB of VRAM: 4K tokens
24 to 48 GB: 32K tokens
48 GB or more: 256K tokens

The same page says coding tools and agents should get at least 64,000 tokens. So on most laptops and gaming cards, the default is a small fraction of what a coding tool expects. By my rough estimate, 4K tokens holds not much more than one medium-sized source file plus your question.

A longer context costs memory, so raise it in steps and check ollama ps after each one.

Windows and macOS: open the Ollama app’s settings and move the context length slider.
Linux: run sudo systemctl edit ollama, add these lines, save, and restart:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=16384"

sudo systemctl restart ollama

One chat only: inside ollama run, type /set parameter num_ctx 16384.

Connecting It to VS Code With Continue

Continue is a free VS Code extension that adds chat, inline edits, and tab autocomplete, and it can talk to Ollama on your machine instead of a cloud service.

In VS Code, open Extensions, search for Continue, and install it.
Pull the two models used below: ollama pull qwen2.5-coder:7b and ollama pull qwen2.5-coder:1.5b.
Open Continue’s config file. It’s ~/.continue/config.yaml on macOS and Linux, and %USERPROFILE%\.continue\config.yaml on Windows.
Replace the models section with this, then reload VS Code:

name: Local Coding
version: 0.0.1
schema: v1

models:
  - name: Qwen2.5 Coder 7B
    provider: ollama
    model: qwen2.5-coder:7b
    roles:
      - chat
      - edit
      - apply
  - name: Qwen2.5 Coder 1.5B
    provider: ollama
    model: qwen2.5-coder:1.5b
    roles:
      - autocomplete

Two models, two jobs. Autocomplete fires constantly as you type, so it needs a small, fast model, and Continue’s autocomplete docs recommend exactly this 1.5B one. Chat can afford to think longer. Keep in mind that both models sit in memory at once.

If Continue complains that the model requires more system memory, it’s probably asking for a bigger context than your machine can hold. Continue’s Ollama page suggests lowering it per model:

    defaultCompletionOptions:
      contextLength: 4096

Put that under the model’s roles list, at the same indentation.

Where a Local Coding Model Falls Short

A light laptop beside a heavy desktop tower, comparing what an Ollama coding model needs for autocomplete and agents

I tried both pieces of this setup: plain Ollama chat and Continue in VS Code. Here’s where it stops being worth it.

Your hardware sets the ceiling

My laptop has an entry-level Nvidia MX450 graphics chip, and that’s where the thirty minutes came from. No setting fixes missing memory. A model that would actually be smart enough to replace my cloud tools, like qwen3-coder:30b, needs 19 GB just to load.

The context is short when you need it long

Real coding means showing the model several files, error logs, and your project’s conventions. On the default 4K, most of that never makes it into view. On a small GPU, raising the context pushes the model onto the CPU. You end up trading speed for memory, or memory for speed.

Agents want hardware most people don’t own

Ollama can start coding agents like Claude Code, Codex, and OpenCode on top of a local model with one command, for example ollama launch claude --model qwen3-coder. Ollama’s Claude Code page tells you to give local models at least 64K of context. Put that together with a 19 GB model and you’re in high-end desktop or big-Mac territory.

Even then, a model that fits at home is a step behind the frontier, which I covered in the local models post. For multi-file refactors, I still reach for a cloud agent, and I still review what it writes. That review is the real job, as I argued in my post on whether AI will replace software engineers.

Where it does pay off

Autocomplete. A small model finishing the line you’re typing is the best fit for local hardware, and it costs nothing per keystroke.
Private code. Ollama’s FAQ says it doesn’t see your prompts when you run models locally. Models with “cloud” in the name are different: they run on Ollama’s servers.
Offline work. Once the models are downloaded, nothing needs a connection.

How to Undo It All

Remove a model: ollama rm qwen2.5-coder:7b. ollama list shows what’s left.
Reset the context length: move the slider back on Windows or macOS. On Linux, sudo systemctl revert ollama removes your override, then restart the service.
Remove Continue: delete the models from config.yaml, or uninstall the extension.
Uninstall Ollama on Windows: use Add or remove programs. If you moved your models with OLLAMA_MODELS, delete that folder yourself, because the uninstaller won’t.
Uninstall Ollama on macOS: delete Ollama from Applications, then remove /usr/local/bin/ollama and the ~/.ollama folder, which holds your models. Ollama’s macOS docs list a few cache folders under ~/Library too.
Uninstall Ollama on Linux: stop and disable the ollama service, delete its service file, the ollama binary and libraries, and /usr/share/ollama, where the models live. Ollama’s Linux docs give the exact commands.

Who Should Bother

If you have a graphics card with plenty of VRAM, or an Apple Silicon Mac with memory to spare, set this up. Local autocomplete and a private chat model for sensitive code are worth an evening.

If you’re on a thin laptop with an entry-level GPU like mine, stick to the 1.5B autocomplete model at most, and keep your expectations low.

And if what you want is an agent that refactors a codebase on its own, pay for a frontier model and spend your effort keeping its bill down. I covered that in my CLI vs GUI AI coding post.

My laptop wrote that hello world eventually. I just don’t plan to ask it for anything bigger.

Frequently Asked Questions

Is running a coding model locally free?

Yes. Ollama is free for local use, Continue is a free extension, and Qwen’s coder models are free to download. You pay in hardware, disk space, and electricity.

Which Ollama coding model should I start with?

qwen2.5-coder:1.5b for autocomplete, since it’s under 1 GB. Add qwen2.5-coder:7b for chat if ollama ps shows it running fully on your GPU.

Why is my local coding model so slow?

Usually because the model doesn’t fit in your GPU’s memory and part of it runs on the CPU. Run ollama ps while it’s loaded. If it shows a CPU share, use a smaller model or a shorter context.

Does my code leave my computer?

Not with local models. Ollama says it doesn’t see your prompts when models run locally. Models with “cloud” in their name run on Ollama’s servers, so check the name before you pull one.