OpenCode with local models: setup and limits

OpenCode is a coding agent for the terminal that doesn’t tie you to a single provider. That means that besides cloud models, you can connect it to one running on your own machine or on a server in your network.

It works, but with caveats. Here is how I set it up and where the limits usually show up, so you know what to expect before spending an afternoon on it.

When it makes sense

The obvious case is code that can’t leave your network: client projects, sensitive data, anything under an agreement. It also helps if you work offline a lot, if you don’t want to pay per token while experimenting, or if you simply want to see how far a local model can go.

Where it makes less sense is long tasks with lots of context, like a big refactor across dozens of files. Small local models struggle there, and the time you save on cost you lose waiting and fixing.

What you need

• OpenCode installed. With npm it’s npm install -g opencode-ai.
• A server that exposes an OpenAI-compatible API. Ollama, LM Studio, llama.cpp’s llama-server and vLLM all do.
• A model that supports tool calls. Without that, the agent can talk but can’t read or edit files.
• Enough memory. If the model doesn’t fit in the GPU it falls back to the CPU and becomes painfully slow.

The server, with Ollama

I’ll use Ollama because it’s the quickest to start. Download a model aimed at code and start the server:

ollama pull qwen2.5-coder:7b
ollama serve

Ollama serves the API at http://localhost:11434/v1. Before going further, there is a detail that trips up many people: by default Ollama uses a small context window, and an agent needs far more than that, because it sends your instructions, the files it reads and the history of the session. With a short window the model forgets what it was doing halfway through.

You can raise it by creating a variant of the model with a larger context:

FROM qwen2.5-coder:7b
PARAMETER num_ctx 32768

Save that as Modelfile and create the model:

ollama create qwen-coder-32k -f Modelfile

A bigger context uses more memory, so adjust the number to what your machine can take.

Configuring OpenCode

Create an opencode.json file in the project root, or a global one in ~/.config/opencode/. You declare the provider as OpenAI-compatible and list the models:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen-coder-32k": {
          "name": "Qwen Coder 32k"
        }
      }
    }
  }
}

The name inside models has to match exactly the one you gave to the model in Ollama. Then open OpenCode in your project, run /models and pick the local one.

The same configuration works for LM Studio, llama.cpp or vLLM. You only change the baseURL and the model name.

What usually goes wrong

• The agent answers but doesn’t do anything. The model doesn’t handle tool calls well. Try another one that explicitly supports them.
• It forgets what it was doing. The context is too short. Go back to the num_ctx step.
• It’s extremely slow. The model is too big for your memory and is running on the CPU. Use a smaller one or a more compressed version.
• Broken edits. Small models often fail to follow the format needed to modify a file. If it happens all the time, the model is too small for this kind of work.

Model on another machine

If you have a server with a GPU on your network, change the address in baseURL, for example to http://192.168.1.50:11434/v1. Ollama only listens on localhost by default, so you have to start it with the OLLAMA_HOST=0.0.0.0 variable.

And don’t expose that port to the internet without authentication. Anyone who can reach it can use your hardware.

How I would use it

Mixing both. Local for small tasks and for code that can’t leave your network, cloud for anything that needs a lot of reasoning or context. OpenCode lets you change model in the middle of a session with /models, so you don’t have to pick one side.

Start with something small, like explaining a function or writing a test, and see how the model behaves before giving it bigger jobs.