Run Your Own AI Assistant on a VPS: Ollama + Open WebUI

Q: How much RAM do I need for Ollama?

The Ollama library does not publish exact RAM requirements. A cautious rule: at least as much free RAM (the 'available' column of free -h) as the model's download size, plus room to work. Check what a loaded model really takes with ollama ps.

Q: Which small model should I start with?

We suggest llama3.2:1b (1.3 GB download in the Ollama library as of 2026-09-24). Smaller options include qwen3:0.6b and gemma3:270m. Tags and sizes change, so check the model's library page first.

Q: Is Open WebUI safe to expose to the internet?

Not directly. Bind its port to 127.0.0.1 in Docker Compose, because Docker-published ports bypass ufw, and serve it through nginx with HTTPS. Create the admin account with a strong password on first start.

Q: What if my VPS is too small for a local model?

Use the same Open WebUI with a provider's API instead. The interface stays on your server, the key lives in a .env file with chmod 600, and the answers come from the provider, paid per token.

Last updated: 2026-09-24 · By Ovanap Team, authors of Linux from Zero (published by Ovanap, Monine AS)

The short answer: yes, you can run a private ChatGPT-style assistant on an ordinary VPS without a GPU. Run Ollama and Open WebUI with Docker Compose, bind the interface port to 127.0.0.1, and publish it through nginx with HTTPS. Pick a small model that fits in the server’s free RAM (for example llama3.2:1b), or connect a provider’s API if the server is too small.

Key facts

Ollama listens on 127.0.0.1:11434 by default. The address changes only through OLLAMA_HOST (Ollama FAQ).
llama3.2:1b is a 1.3 GB download in the Ollama library as of 2026-09-24 (ollama.com). The library does not publish exact RAM requirements.
Open WebUI’s docs require proxy_buffering off; behind nginx (buffering garbles streamed answers), WebSocket upgrade headers, and the CORS_ALLOW_ORIGIN variable (Open WebUI nginx guide).
Docker-published ports bypass ufw (Docker docs). That is why the compose file binds 127.0.0.1:3000:8080.
A GPU is optional. It buys speed, not the ability to run a model. On CPU, small models answer slowly but work.

When you use an AI chat website, three things are not yours: the rules (what the model may do, and when the service changes), the data (your questions go to someone else’s server), and the bill. You can move the whole conversation onto a server you control: an open model running under Ollama (a program that downloads open language models and runs them on your machine), a chat interface from Open WebUI (an open-source, ChatGPT-style web interface), both in Docker containers, behind nginx (a web server acting as a reverse proxy: it accepts HTTPS connections and forwards them to the internal service).

This guide shows the setup on an Ubuntu Server 26.04 LTS VPS, with honest expectations about RAM and speed, and the one Docker mistake that exposes the whole thing to the internet. It assumes you already have a secured server (see our VPS checklist), nginx installed, and a domain.

First, will a model fit? RAM, not a GPU

A model is billions of numbers (weights), and all of them have to sit in memory while it answers. Think of the disk as a cupboard and RAM as your desk: the model’s table of numbers must fit on the desk.

A graphics card is not required. A GPU is a stadium of simple workers doing identical multiplications at once, so it makes answers much faster. Without one, the processor does the work. It’s slower, but it works. Quantization is what makes that practical: the same model stored at lower precision, like a compressed photo, with tags such as Q4 or Q8.

Measure before you choose:

free -h

Look at the available column. The rule: the model must fit into available with room to spare. The Ollama library doesn’t publish exact RAM requirements, so we don’t invent any. Use the download size as a guide, and after loading a model, ollama ps shows in its SIZE column what it really takes.

Small models from the Ollama library (download sizes as listed on 2026-09-24):

Model	Download size
`gemma3:270m`	292 MB
`qwen3:0.6b`	523 MB
`gemma3:1b`	815 MB
`llama3.2:1b`	1.3 GB
`qwen3:1.7b`	1.4 GB
`llama3.2:3b`	2.0 GB

We suggest starting with llama3.2:1b. Sizes and tags change, so we keep the list current in the models appendix. Remember that Open WebUI and the system need memory too. If available is smaller than the model, pick a smaller one, or use path B below.

Set two honest expectations. Without a GPU, answers appear as if someone were dictating them: normal, not a fault. And if a model barely fits, the server sinks into swap and “thinks” for minutes: the cure is a smaller model.

Two paths to the same assistant

Path A: a local model. Ollama runs the model on your VPS. Questions stay on your server; memory and CPU set the limits.
Path B: a provider’s API. Open WebUI runs on your server, but answers come from a provider (for example OpenAI) through an API key you control. You pay per token. Use it when the VPS is too small or you want a stronger model.

Both end in the same place: a private chat interface in your browser.

Step 1: Install Docker Engine

Install Docker Engine from Docker’s official repository, not the docker.io package. The exact lines mirror docs.docker.com and are in our Docker appendix. Check:

docker --version
sudo docker run hello-world

Step 2 (optional): Try Ollama in the terminal first

Want to see a model answer before building anything? The official install script:

curl -fsSL https://ollama.com/install.sh | sh

This downloads a script and runs it immediately, so use it only from a source you trust. To read it first, download it with -o /tmp/ollama-install.sh, open it in less, then run it. A manual install without the script is in the Ollama appendix.

ollama run llama3.2:1b

Ask a question, leave with /bye. The same model also answers programs through a local API on port 11434:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2:1b",
  "prompt": "Write one sentence about Linux.",
  "stream": false
}'

The answer is in the "response" field of the JSON. By default Ollama listens only on 127.0.0.1:11434. Keep it that way: setting OLLAMA_HOST=0.0.0.0 opens it to the network with no password.

The Docker setup below runs its own Ollama, so once you’re done experimenting, clean this one up: ollama rm llama3.2:1b and sudo systemctl disable --now ollama.

Step 3: Ollama + Open WebUI with Docker Compose

Docker Compose describes several containers in one YAML file and starts them together with one command.

mkdir -p ~/ai-service && cd ~/ai-service
nano compose.yaml

services:
  ollama:
    image: ollama/ollama
    restart: unless-stopped
    volumes:
      - ollama-data:/root/.ollama

  ui:
    image: ghcr.io/open-webui/open-webui:main
    restart: unless-stopped
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - CORS_ALLOW_ORIGIN=https://ai.example.com
    volumes:
      - ui-data:/app/backend/data
    ports:
      - "127.0.0.1:3000:8080"
    depends_on:
      - ollama

volumes:
  ollama-data:
  ui-data:

Replace ai.example.com with your own subdomain. Then:

docker compose up -d
docker compose exec ollama ollama pull llama3.2:1b
docker compose ps

What the key lines do:

Volumes keep models, chats, and settings when containers are updated or recreated.
OLLAMA_BASE_URL=http://ollama:11434: containers reach each other by service name, because each container has its own “localhost.”
CORS_ALLOW_ORIGIN: recommended by Open WebUI’s docs behind a reverse proxy, or the WebSocket connection can fail.
restart: unless-stopped: everything comes back after a reboot. Test it for real with sudo reboot, then docker ps.

Step 4: The port mistake that exposes everything

This is the most important line in the file: "127.0.0.1:3000:8080".

Ports that Docker publishes bypass ufw. Write "3000:8080" without the address, and your chat interface is open to the whole internet, even though sudo ufw status shows nothing. With 127.0.0.1: in front, port 3000 exists only inside the server. Prove it from your own computer:

curl -I --max-time 10 http://203.0.113.10:3000

It must end in Connection timed out. Here, an error is the good result.

Step 5: nginx and HTTPS in front

Create /etc/nginx/sites-available/ai:

server {
    listen 80;
    listen [::]:80;
    server_name ai.example.com;

    client_max_body_size 50M;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_buffering off;
        proxy_cache off;
    }
}

The Upgrade/Connection lines let the live WebSocket stream through. proxy_buffering off stops nginx from re-chunking streamed answers (otherwise you may see stray ## and **). client_max_body_size allows document uploads larger than nginx’s 1 MB default. Enable and secure it:

sudo ln -s /etc/nginx/sites-available/ai /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx
sudo certbot --nginx -d ai.example.com

The subdomain needs its own DNS A record pointing to your server. Open https://ai.example.com and immediately create the admin account with a strong password. More detail is in the Open WebUI appendix.

Path B: when the VPS is too small

Drop the ollama service and depends_on, and give Open WebUI a provider’s API instead. The key goes in ~/ai-service/.env (one line, OPENAI_API_KEY=...) protected with chmod 600, never in the compose file, Git, or a screenshot. In the compose file you write only ${OPENAI_API_KEY}; docker compose reads .env by itself. Open WebUI stores these settings on first start, so later changes go through Settings → Admin → Connections. The complete file, and notes on other providers, are in the API appendix. Set a spending limit in the provider’s dashboard.

Three safety rules

A key stays a key. Only in .env with chmod 600. If it leaks, revoke it at the provider.
The interface is not a public website until it sits behind nginx with HTTPS and a strong admin password.
A model’s answer is not a fact. Small local models are wrong more often and sound just as confident. Read any command one suggests before you run it.

Frequently asked questions

Can I run Ollama on a VPS without a GPU? Yes. A GPU buys speed, not the ability to run a model. On a CPU-only VPS, small models work, just slowly.

How much RAM do I need for Ollama? The Ollama library does not publish exact requirements. At least the model’s download size in available RAM, plus room to work; check real usage with ollama ps.

Which small model should I start with? llama3.2:1b (1.3 GB as of 2026-09-24), or smaller qwen3:0.6b and gemma3:270m. Check the library page first.

Is Open WebUI safe to expose to the internet? Not directly. Bind it to 127.0.0.1, serve it through nginx with HTTPS, and set a strong admin password.

What if my VPS is too small for a local model? Use Open WebUI with a provider’s API (path B), with the key in a .env file.

Build it with every step explained

This setup is the destination of Linux from Zero. The book gets you there from zero: the terminal, a secured VPS, nginx with HTTPS, backups, Docker, then this AI service in a lab, with each command explained and a way back for every risky step. Because models, install lines, and VPS offers change fast, the book points to an online appendix we keep current.

Read the free sample (no sign-up), then get the book if it suits you.