The best mini-PC for a home LLM server (and when a mini-PC is the wrong call)
Heads up: some links in this article are affiliate links. If you buy through them I may earn a small commission at no extra cost to you. As an Amazon Associate I earn from qualifying purchases. I only recommend gear I'd run myself.
In a previous post I set up a local LLM on a desktop GPU and reached it from anywhere over Tailscale. The obvious follow-up question is: “I don’t want a big tower humming in the corner — what mini-PC would run this?”
It’s a good question with an unobvious answer, because the spec that sells a mini-PC (cores, clock speed) is not the spec that runs a language model. Buy on the wrong number and you’ll spend $700 on a box that generates text slower than you can read it. Here’s the version I wish someone had told me first.
The one number that decides everything: memory bandwidth
LLM inference is not compute-bound the way gaming or video encoding is. For every token it generates, the model has to stream its entire set of active weights out of memory. That makes memory bandwidth, not raw CPU speed, the thing that sets how fast tokens come out.
This is why a plain mini-PC running a model on the CPU feels broken: dual-channel DDR5 gives you maybe 80–90 GB/s, and the model chokes on that pipe. A discrete GPU works because its VRAM runs at hundreds of GB/s to over a terabyte per second. So the real question for a small box isn’t “how many cores” — it’s “how do I get a lot of fast memory the model can actually reach?” (how much VRAM a local LLM needs covers the sizing.)
There are only three honest ways to do that in something small and quiet.
Option 1 — Apple Silicon Mac mini (the turnkey pick)
Apple’s M-series chips use unified memory: the CPU and GPU share one fast pool, and on the Pro/Max tiers that pool has serious bandwidth (roughly 120 GB/s on the base chip, ~270 GB/s on the Pro). The whole RAM figure effectively becomes usable “VRAM,” so a Mac mini with 32–64GB can hold models a 16GB discrete GPU simply can’t fit.
It’s the least-hassle option by a wide margin: small, near-silent, sips power at idle, and ollama runs natively with Metal acceleration. If you want to buy one thing, plug it in, and be serving models over your tailnet the same evening, this is it.
- Get the Pro chip if you can — the bandwidth jump is what you’re paying for, not the extra CPU cores.
- Buy as much unified memory as the budget allows. On Apple Silicon, memory is your model-size ceiling; you cannot add more later.
The turnkey small-form LLM box — get the Pro tier and the most unified memory you can, because it can't be upgraded later:
See Mac mini configs on Amazon ↗Option 2 — an x86 mini-PC with a strong iGPU (the tinkerer’s value pick)
The interesting development on the PC side is the newer AMD Ryzen mobile chips (the Ryzen AI / 8000-series parts with Radeon 780M/890M graphics) that let you allocate a large slice of system RAM to the integrated GPU. Load a box like that with 64GB of fast DDR5, hand a big chunk to the iGPU, and you can run mid-sized models on ROCm or Vulkan at usable speeds — for well under the price of a comparable Mac.
It’s more fiddly (driver setup, deciding the VRAM split in BIOS), and it won’t match a discrete GPU token-for-token, but it’s the best price-per-capability in a small box if you enjoy the tuning.
- Buy the fastest RAM the board supports and fill both channels — dual-channel is not optional here; single-channel halves your bandwidth and your speed.
- 64GB is the sweet spot so you can give the iGPU a generous allocation and still run the OS comfortably.
A Ryzen AI mini-PC with a capable iGPU — pair it with a fast dual-channel kit:
See Ryzen AI mini-PCs on Amazon ↗Fast DDR5 in both channels is what makes the iGPU route worthwhile:
See 64GB DDR5 SO-DIMM kits on Amazon ↗Option 3 — a small PC plus an external GPU over Oculink
If you want real discrete-GPU speed but still hate the idea of a full tower, a growing number of mini-PCs and handhelds now expose an Oculink port — essentially PCIe on a cable. Plug in an external GPU dock, drop a real card into it, and the small box drives a proper GPU with a fraction of the bandwidth penalty an old Thunderbolt eGPU had.
This is the most flexible path: start CPU-only, add the GPU when you’re ready, upgrade the card later without replacing the computer. The trade-off is that it stops being tidy — you now have a box plus a dock plus a card, which is halfway back to a tower.
An Oculink eGPU dock turns a small PC into a real GPU host — add the card of your choice:
See Oculink eGPU docks on Amazon ↗When a mini-PC is the wrong call
Be honest with yourself about the goal. If you want to run large models (30B and up) fast, no mini-PC is the value play — you’ll pay a premium for the small form factor and still be memory-starved. A used tower with a 16GB discrete GPU, from the home-GPU setup in the earlier post, will beat every option here on raw tokens-per-dollar. The mini-PC wins on silence, power draw, and desk space — not on peak performance.
What I’d actually buy
- Want it to just work, tonight: Mac mini, Pro chip, maximum unified memory you can afford.
- Want the most capability per dollar and enjoy tuning: a Ryzen AI mini-PC with 64GB of fast dual-channel RAM.
- Want small and real GPU speed, and don’t mind a dock on the desk: a mini-PC with Oculink plus an external GPU.
- Want maximum speed and don’t care about size: skip this list — build the tower.
Whichever you pick, the setup on top is the same: install ollama, put both the box and your laptop on Tailscale, and point an OpenAI-compatible client at its tailnet address — exactly the walkthrough from last time. The hardware just decides how fast the answers come back. If you would rather have one always-on box for everything else too, see mini PCs for an always-on homelab; and if you are unsure which model server to run, Ollama vs llama.cpp vs vLLM compares them.