Using a Mac mini as an AI server
Using a Mac mini as an AI server means three jobs: models, agents and apps. Which memory size handles which, and what to keep isolated from your files.
The Mac mini everyone bought in the spring
Using a Mac mini as an AI server is the reason a lot of them exist. In the spring of 2026, an agent project sent people to the Apple Store in numbers that made the mini hard to buy. The agent turned out to be less useful than the thread promised. The mini did not.
So the machine is on a shelf, and the question is what an AI server actually is, because the phrase covers three different jobs. A mini is excellent at some of them and honestly unsuited to others. Knowing which is which is the difference between a useful box and a $600 paperweight.
Three jobs an AI server does
Sort what you want into these before you install anything, because they want different memory, different software and different exposure to the internet.
- Running modelsOllama, LM Studio or MLX serving a language model on your own hardware. This is the job the mini is famous for, and the one where memory size decides everything.
- Hosting agentsOpenClaw, Claude Code, a Hermes or n8n workflow: a program that sits there all day, watches for messages or schedules, and acts. This wants uptime and isolation far more than it wants a fast chip.
- Serving AI appsA chat front end for your family, an API that calls a hosted model, a bot in a Discord. Ordinary web hosting that happens to talk to a model somewhere. Any mini handles this.
Why a Mac mini as an AI server makes sense
Two numbers explain the mini's reputation. Apple's August 2026 environmental report measures the current model at 3.0 W idle, so leaving it on costs a few dollars a year. And Apple silicon shares one pool of memory between the CPU and the GPU, so a model that fits in memory runs on the GPU without a graphics card.
That second point is why a 32 GiB mini can hold a model that would need a $1,500 graphics card elsewhere. It is not the fastest way to run a model. It is the quietest, cheapest, always-on way, and for one person talking to a model, that is the trade that matters.
Apple: Mac mini product environmental report (August 2026, PDF) ↗
Which Mac mini for which model
Memory is the whole decision, because a model has to fit in it with room for macOS. A rule of thumb that holds up: a model at 4-bit quantisation needs about half its parameter count in gigabytes, plus a couple more for the working context. macOS wants four to six gigabytes for itself before anything starts.
- 16 GiBModels up to about 8 billion parameters run well, and 12 to 14 billion run with the context kept short. Enough for a capable local assistant, code completion, and summarising your own documents. Not enough to also run much else at the same time.
- 24 GiBThe comfortable size for 14 billion parameter models with a real context window, and the smallest size where a model and a couple of services coexist without the machine swapping.
- 32 GiBModels around 30 billion parameters at 4-bit fit, and this is where local models stop feeling like a toy. Also the point where you can run a model and a Linux server side by side and forget about memory.
- 64 GiB and upThe 70 billion parameter class becomes possible. It is slow on a base chip, because speed comes from memory bandwidth, and a Pro chip roughly doubles it. If this is the goal, buy the Pro.
Speed is bandwidth, not cores
Generating text streams the whole model through memory for every token. So tokens per second is roughly memory bandwidth divided by model size. A 5 GB model on a base M4 at about 120 GB/s gives around 20 tokens a second, which reads faster than you do. The same model on an M4 Pro, at roughly 270 GB/s, more than doubles that. Extra CPU cores barely move it.
Those two figures are the M4 generation, because that is what we measured on. Every newer chip raises the bandwidth and so raises the tokens per second, but none of it changes the shape of the decision: look up your own chip's memory bandwidth, divide by the size of the model on disk, and you have your answer within a few tokens a second.
Install a model and talk to it in five minutes
Ollama is the shortest path, because it installs as one command, runs on the GPU through Apple's own framework, and speaks an API that almost every chat app and agent already understands. Everything later in this guide assumes it. If you prefer a window, LM Studio does the same job with a graphical interface and the same kind of API.
Start with an 8 billion parameter model whatever memory you have. It downloads in minutes, answers quickly, and tells you whether the mini is doing what you expect before you commit to a 20 GB download.
- Install Ollama with Homebrew, or download the app from ollama.com. Both give you the same
ollamacommand. - Pull a model. The first pull is the slow part, so pick a small one to prove the setup.
- Ask it something from the terminal. Then ask the API the same thing, because the API is what everything else will use.
1. Install, and start it as a service that survives logout
brew install ollama
brew services start ollama2. Pull a small model
ollama pull llama3.1:8b3. Talk to it, then talk to its API
ollama run llama3.1:8b "Explain unified memory in one sentence."
curl http://localhost:11434/api/generate -d '{"model":"llama3.1:8b","prompt":"Say hello.","stream":false}'The API is OpenAI-shaped, which is why it matters
Ollama also answers at /v1/chat/completions in the same format as OpenAI's API. So any tool that takes an OpenAI base URL, which is most of them, can point at your mini instead. That one fact turns the mini from a chat toy into infrastructure.
Keep the model loaded
By default Ollama unloads a model after five minutes idle, so the first reply after a quiet spell takes several seconds while it reloads. On a machine that exists to answer, set the keep-alive to forever. It costs memory you were not using anyway.
Keep the last model in memory permanently, then restart the service
launchctl setenv OLLAMA_KEEP_ALIVE -1
brew services restart ollamaOpen the API to your other devices
Out of the box Ollama listens only to the mini itself, which is the right default and useless for a server. Open it up in three stages, and stop at whichever one you actually need. Each stage exposes more, so each one needs more care.
One warning first, because it decides everything below. Ollama has no authentication of its own. Anyone who can reach the port can use your model and your electricity, so never put port 11434 directly on the internet. Put it on your network, then behind a VPN, and only ever on the public internet behind something that checks a key.
Stage one: your home network
Tell Ollama to listen on every interface and restart it. From a laptop on the same Wi-Fi, replace localhost with the mini's name or address and the same requests work. This is enough for a chat app on your phone at home.
Listen on all interfaces, persistently
launchctl setenv OLLAMA_HOST 0.0.0.0
brew services restart ollamaFrom another machine in the house
curl http://your-mini.local:11434/api/tagsStage two: from anywhere, privately
Install Tailscale on the mini and on the devices you carry. Each gets a stable private address that works from a café or a hotel, nothing is exposed publicly, and carrier-grade NAT does not matter. Then let Tailscale put HTTPS in front of the port, so apps that insist on a secure address are happy.
This is the stage most people should stop at. Your phone, your laptop and your own agents reach the model. Nobody else can.
Serve Ollama over HTTPS to your own devices only
tailscale serve --bg 11434Stage three: the public internet, behind a key
Sometimes the caller is not you: a hosted agent, a webhook, a friend's app. Then the API needs a public address, and the address needs a lock. The simplest lock is a reverse proxy that rejects any request without a secret header. Caddy does it in four lines and handles the certificate for you.
Point a domain at the mini, or run this behind a tunnel or a relay if you have no public address. Then give the token only to callers you trust, and rotate it when one leaves.
Caddyfile: a bearer token in front of Ollama, with HTTPS from Let's Encrypt
ai.example.com {
@noauth not header Authorization "Bearer REPLACE_WITH_A_LONG_RANDOM_STRING"
respond @noauth 401
reverse_proxy localhost:11434
}Install and run it
brew install caddy
caddy run --config ./CaddyfileMac mini server remote access: port forwarding, VPNs, tunnels and relays compared ↗
Put a chat app in front of it
A terminal is fine for you and no good for anyone else. Open WebUI is the chat interface most people land on: it looks like the hosted chat apps, keeps history, supports several users, and talks to Ollama with one setting. It runs in Docker, so install Docker Desktop or OrbStack on the mini first.
Once it is up, it is a website on port 3000. On the home network that is http://your-mini.local:3000. Behind Tailscale it is private and reachable from anywhere. Behind a public address it is the thing your family uses on their phones, and the model behind it never needs to be exposed at all.
Open WebUI, pointed at the Ollama running on the mini
docker run -d -p 3000:8080 \
-e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
-v open-webui:/app/backend/data \
--name open-webui --restart always \
ghcr.io/open-webui/open-webui:mainHost an agent without handing it your Mac
An agent is a program with a shell, keys, and a habit of acting on what it reads. Run it in your own macOS account and it has your documents, your keychain and your browser sessions. The reason people bought a second Mac was to keep it away from all that. You do not need a second Mac. You need a second machine, and the mini can be both.
The cheap version: a separate user
Create a Standard macOS account with no admin rights, sign in to it, and install the agent there. It cannot read your files or your keychain. It can still see the whole network and every app on the machine, so treat this as a fence, not a wall.
The proper version: a Linux machine on the mini
A Linux virtual machine is a wall. The agent gets its own disk, its own users and its own network, and nothing it does touches macOS. UTM and Lima are both free and both run Linux on Apple silicon well. Give the VM a fixed slice of memory, keep the model outside it, and the two coexist.
From inside the VM, the model on macOS is one address away. The VM's default gateway is the mini, so ask for that and point the agent at it.
From inside the Linux VM: find the mini, then ask Ollama what it has loaded
HOST=$(ip route | awk '/default/ {print $3}')
curl http://$HOST:11434/api/tagsWhy the model stays outside the VM
A Linux VM on Apple silicon does not get the GPU. A model inside it runs on the CPU and is several times slower. That is a limit of the platform, not of any tool, so the split is always the same: the model on macOS where the GPU is, the agent in Linux where it cannot reach your files, and the two talking over the mini's own network.
Keeping it on, and keeping it honest
A machine that answers a chat at 3 a.m. has to be awake at 3 a.m. Set sleep to never, tell it to restart after a power cut, and make sure your services start without anyone logging in. A mini that waits at the FileVault screen after an update is a server that quietly stopped being one.
- Never sleep
sudo pmset -a sleep 0 disksleep 0 autorestart 1covers sleep and the power cut in one line. - Start before login
brew services start ollamaruns at your login.sudo brew services start ollamamakes it a system service that runs at boot with nobody logged in, which is what a server wants. The Docker app and the Ollama app are login items, so either enable automatic login or use the service form. - Watch the memoryA model that fits alone may not fit beside a Linux VM and a media app. macOS also caps how much unified memory the GPU may use, at roughly two thirds to three quarters. Keep the model inside that, and the mini stays predictable.
- Keep it reachableSleep, the login screen and a changed address are the three ways it goes quiet while you are away. Each has a fix in the remote access guide linked above.
What a mini is not
It does not train models. It does not serve a model to a crowd; one person at a time is the shape it is built for. And it does not replace a hosted frontier model for hard reasoning, so most working setups use both: the local model for private and cheap, the hosted one for difficult. That split is fine. It is what the mini is for.
Where MacServe fits, today and later
MacServe does not run models today, and this guide does not need it to. What MacServe does is the server half of the picture above: it creates the Linux machine on the mini for the agents and the chat app, gives it a public HTTPS address and a public SSH port with keys, and brings it back after a reboot with nobody logged in. The model stays on macOS, where the GPU is, and the Linux server reaches it over the mini's own network exactly as described above.
The model's own API is the next thing on the list. A later MacServe release will run Ollama natively on macOS, with the GPU, and publish its API through the same relay behind credentials you control, so stage three above becomes a switch rather than a Caddyfile. Until then, everything on this page works without us, and the stage two setup is the one we would run ourselves.