How to Set Up Local AI Coding (No Cloud Required)

Running AI coding tools locally means your code never leaves your machine. No API costs, no internet required, no privacy policy to worry about. This guide covers every step from hardware assessment to a working IDE setup.

Why Go Local?

The case for local AI is strongest in these scenarios:

• Proprietary codebase: Client contracts, NDAs, or internal policy prohibit sending code to third-party APIs
• High volume usage: If you're running agents that make hundreds of API calls per day, local inference can be cheaper than cloud after hardware amortization
• Offline development: Planes, trains, remote sites, corporate networks with outbound restrictions
• Latency sensitivity: Local inference on a good GPU can be faster than round-trip API calls for small models
• Privacy by principle: Some developers simply prefer keeping their work on their own hardware

The tradeoff: local models are still behind frontier models (GPT-4o, Claude 3.5 Sonnet) on complex reasoning and very large codebase tasks. For most daily coding tasks — completions, simple refactors, test writing — the gap has narrowed enough to be practical.

---

Hardware Requirements

Your GPU is the primary constraint. Here's what you can run at each tier:

CPU-only: Works with 7B models at 3–8 tokens/second. Usable for chat with patience; not practical for real-time completion.

Minimum practical setup: Any NVIDIA GPU with 8GB+ VRAM, or Apple Silicon with 16GB+ unified memory.

---

Model Benchmarks

Choose your model based on VRAM budget and task type:

Note: HumanEval and MBPP scores are for base/instruct variants. Actual completion quality in IDE use varies — these benchmarks measure standalone code generation. For comparison: GPT-4o scores ~90.2% on HumanEval, Claude 3.5 Sonnet ~92%.

---

Step 1: Install Ollama

Ollama is the de facto standard for running local LLMs. It handles model downloads, quantization selection, and exposes a REST API that most AI coding tools can speak to.

# macOS/Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows: download installer from ollama.com/download
# Then verify install:
ollama --version

Pull your first model:

# Best option for 12GB+ VRAM (2.5GB download, Q4 quantized):
ollama pull qwen2.5-coder:7b

# Best option for 16GB+ VRAM (10GB download):
ollama pull deepseek-coder-v2:16b

# Best option for 24GB+ VRAM (20GB download):
ollama pull qwen2.5-coder:32b

# Verify it works:
ollama run qwen2.5-coder:7b "Write a Python function to flatten a nested list"

Ollama starts a server on http://localhost:11434 automatically when you run a model. It stays running in the background.

---

Step 2: Set Up Continue in VS Code

Continue is the best open-source AI coding plugin, and it connects directly to Ollama.

# Install from VS Code Extensions panel:
# Search: "Continue - Codestral, Claude, and more"
# Or via CLI:
code --install-extension Continue.continue

Edit the Continue config file (~/.continue/config.json):

{
  "models": [
    {
      "title": "Qwen2.5 Coder 7B (Local)",
      "provider": "ollama",
      "model": "qwen2.5-coder:7b",
      "contextLength": 32768
    },
    {
      "title": "DeepSeek Coder V2 16B (Local)",
      "provider": "ollama",
      "model": "deepseek-coder-v2:16b",
      "contextLength": 163840
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5 Coder 7B (Autocomplete)",
    "provider": "ollama",
    "model": "qwen2.5-coder:7b"
  },
  "contextProviders": [
    { "name": "code" },
    { "name": "docs" },
    { "name": "diff" },
    { "name": "terminal" },
    { "name": "open" }
  ]
}

Reload VS Code. You should see a Continue panel on the left with your local models available.

---

Step 3: Set Up Aider for Local Agentic Coding

Aider is the best local-compatible coding agent. It runs in your terminal and connects to Ollama.

pip install aider-chat

# Start with DeepSeek Coder V2 (if you have it pulled)
aider --model ollama/deepseek-coder-v2:16b

# Or with Qwen 32B for best local quality:
aider --model ollama/qwen2.5-coder:32b

# Useful flags:
aider --model ollama/deepseek-coder-v2:16b \
  --no-auto-commits \       # Review before committing
  --watch-files \           # Auto-add changed files
  --map-tokens 2048          # How much of codebase map to include

Once in the Aider REPL:

> /add src/api/users.ts src/services/userService.ts
> Add input validation to the createUser endpoint using zod

---

Step 4: Configure JetBrains IDE (Optional)

If you use IntelliJ, PyCharm, or another JetBrains IDE:

1. Install the Continue JetBrains plugin from the marketplace
2. The configuration is shared with VS Code (~/.continue/config.json)
3. Alternatively, install LLM Plugin for a lighter-weight option

---

Cost Analysis: Local vs. Cloud

Scenario: Developer making 500 AI queries/day (completions + chat + agent tasks)

Cloud (Copilot Pro + Claude API):
  Copilot Pro:        $10/mo
  Claude Sonnet API:  ~$25/mo at moderate agent use
  Total:              ~$35/mo = $420/year

Local (one-time hardware + electricity):
  RTX 4080 GPU:       $700 (amortized over 3 years = $233/year)
  Electricity (150W): ~$10/mo = $120/year
  Ollama + software:  $0
  Total year 1:       $820 (hardware + running costs)
  Total year 2+:      ~$120/year

Break-even: ~20 months for a heavy user paying $35/month in cloud costs.
For a $20/month cloud user: break-even is ~35 months.

The math favors cloud for most individual developers. Local makes sense if: (1) you have strong privacy requirements that cloud tools can't satisfy, (2) you already have suitable hardware, or (3) you're running a team with multiple developers.