XDA published a piece this month arguing that a smaller local AI stack is a more productive local AI stack. The author had accumulated the usual sprawl: two model runners, three UIs, a vector database no one used, and a folder of adapters they had forgotten about. Cutting back to a minimal set made the setup easier to maintain and easier to reach for. We tested eight of the best apps for a minimalist local AI stack on desktop, on a MacBook Pro M3, a Windows workstation with an RTX 4070, and a Debian box with an AMD 7800 XT, focused on how each one behaves as the only tool in the stack rather than one of many.
What to look for in a minimalist local AI app
- Single-binary install or close to it. A tool that is a
brew installor an.msibeats a tool that needs Docker plus Python plus a config file. - Model management that stays out of the way. Downloading a GGUF should be one command. Deleting one should not require finding a hidden folder.
- API compatibility. An OpenAI-compatible endpoint is what lets you point every existing tool at your local model without rewriting either side.
- GPU and CPU auto-detection. The stack should figure out whether you have Metal, CUDA, ROCm, or CPU-only and use the right one. Manual backend selection is the sprawl these tools were meant to remove.
- RAM discipline. A tool that idles at 2 GB of RAM to serve a model that fits in 8 GB is doing something wrong.
Quick comparison
| App | Best for | Platforms | Free plan | Starting price | Standout feature |
|---|---|---|---|---|---|
| llama.cpp | The reference runtime | Windows, macOS, Linux | Free, open source | Free | Portable single-binary inference |
| llamafile | One file, no install | Windows, macOS, Linux | Free, open source | Free | Model plus runtime in one executable |
| Ollama | Simplest CLI plus API | Windows, macOS, Linux | Free, open source | Free | ollama run and you have an OpenAI endpoint |
| Jan | Native desktop UI with API | Windows, macOS, Linux | Free, open source | Free | Cross-platform desktop chat |
| KoboldCpp | Single-binary UI plus API | Windows, macOS, Linux | Free, open source | Free | GGUF runner with built-in web UI |
| Msty | Polished paid UI on local models | Windows, macOS, Linux | Free tier | Msty subscription for advanced features | Multi-model chat interface |
| GPT4All | Simplest desktop-only chat | Windows, macOS, Linux | Free, open source | Free | Beginner-friendly, no CLI |
| LM Studio | Full-featured local runner | Windows, macOS, Linux | Free for personal | Business licensing | Model discovery UI, wide backend support |
The apps
1. llama.cpp — Best for the reference runtime everyone else builds on
llama.cpp is the runtime that most of this list wraps in one way or another. It is a portable, dependency-free C++ inference server that runs GGUF models on CPU, Metal, CUDA, and ROCm. As a minimalist stack on its own, llama-server gives you an OpenAI-compatible endpoint, and llama-cli gives you a REPL. That is enough for most local AI work.
Where it falls short: No polished UI. You are compiling or downloading a release binary. Model management is manual.
Pricing:
- Free: Fully open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: github.com/ggerganov/llama.cpp
Bottom line: llama.cpp is the pick when you want the fewest moving parts and are comfortable in a terminal.
2. llamafile — Best for a runtime and a model in one executable
llamafile by Mozilla’s Ocho project bundles a model and the llama.cpp runtime into one Cosmopolitan-libc binary that runs on Windows, macOS, and Linux without install. Download the file, chmod it, run it. It opens a browser UI and exposes an OpenAI-compatible API on localhost. It is the most literal interpretation of “minimalist local AI” on the list.
Where it falls short: Portable single-binary files can hit host security tooling. Downloading a new model means downloading a new llamafile.
Pricing:
- Free: Fully open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: github.com/Mozilla-Ocho/llamafile
Bottom line: llamafile is the pick when the install must be one file and one file only.
3. Ollama — Best for the simplest CLI plus API workflow
Ollama wraps llama.cpp with the cleanest install and command experience of anything on this list. ollama pull llama3.2:8b downloads a model. ollama run llama3.2:8b opens a chat. ollama serve exposes an OpenAI-compatible endpoint on port 11434 by default. The Modelfile format lets you pin system prompts and parameters with a single text file.
Where it falls short: The default install runs the server as a background service you may not want. Manual model quantization is limited compared to raw llama.cpp.
Pricing:
- Free: Fully open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: ollama.com
Bottom line: Ollama is the pick when a one-command install and a one-command run are the minimum bar.
4. Jan — Best for a native desktop UI you own
Jan is a from-scratch desktop app for local models, open source, with an OpenAI-compatible API server built in. It is what you install when you want a real desktop chat UI without the LM Studio surface area. The team ships builds for all three OSes and keeps the model discovery flow simple.
Where it falls short: Younger project than LM Studio or Ollama. Some of the advanced features (specialized backends, deep model customization) lag.
Pricing:
- Free: Fully open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: jan.ai
Bottom line: Jan is the pick when a proper desktop UI is the deciding factor.
5. KoboldCpp — Best for a single-binary UI plus API on a portable target
KoboldCpp started as a fork of llama.cpp aimed at the KoboldAI community, and it has grown into a single-binary GGUF runner with a browser UI, an API endpoint, image and audio input support, and no install step. Drop the binary and a GGUF next to it, run one command, and everything is up.
Where it falls short: Some of the specialized features (Horde integration, KoboldAI-specific prompts) do not matter for a general chat use case.
Pricing:
- Free: Fully open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: github.com/LostRuins/koboldcpp
Bottom line: KoboldCpp is the pick when you want a single binary that gives you both a UI and an API without a heavier app.
6. Msty — Best for a polished commercial UI on top of local models
Msty takes the LM Studio approach and produces the cleaner UI. Chat side-by-side with multiple models, connect to local Ollama or Jan endpoints, and organize conversations in folders. The free tier is generous; the paid tier is what unlocks the multi-user and workspace features.
Where it falls short: Closed-source. Not on the minimalist end of “install one open-source binary.” For a solo user who wants polish over principles, that is fine.
Pricing:
- Free: Full core UI
- Paid: Msty subscription for advanced workspace features
Platforms: Windows, macOS, Linux
Download: msty.app
Bottom line: Msty is the pick when the UI polish is worth trading the open-source constraint for.
7. GPT4All — Best for a beginner-friendly desktop chat
GPT4All by Nomic is the friendliest way to get a local LLM running for someone who does not want to touch a terminal. Install the app, browse a curated model list, pick one, chat. It also exposes a local API and integrates with the Nomic Atlas for document-based RAG if you decide to go beyond chat.
Where it falls short: The curated model list is a small subset of what is available on Hugging Face. Advanced users grow out of it.
Pricing:
- Free: Fully open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: gpt4all.io
Bottom line: GPT4All is the pick for a first-time local AI user who wants a desktop app that just works.
8. LM Studio — Best for a full-featured local runner
LM Studio is the maximalist end of “still a single desktop app.” Model discovery from Hugging Face, multi-backend support (llama.cpp, MLX on Apple Silicon), a full chat UI, an API server, and a model configuration surface that exposes everything. It is what people default to when Ollama’s simplicity is too limited.
Where it falls short: Not open source. Larger install footprint than the picks above. The maximalist feature surface is the opposite of the minimalist stack the XDA piece was arguing for, but it is included here as the reference point for what happens when you refuse to trim.
Pricing:
- Free: Personal use
- Paid: Business licensing for commercial deployments
Platforms: Windows, macOS, Linux
Download: lmstudio.ai
Bottom line: LM Studio is the pick when a full-featured local runner is worth the larger install than any of the above.
How to pick the right one
- The minimum viable stack: Ollama. Install it, pull a model, done. Everything else is optional.
- If you would rather have one file and no install: llamafile.
- If you want the reference runtime with no wrapper: llama.cpp directly.
- If you want a desktop chat UI and open source: Jan.
- If you want a single binary that is also a UI: KoboldCpp.
- If polish matters more than open source: Msty on top of an Ollama endpoint.
- If you have never installed a local model before: GPT4All.
- If you have outgrown Ollama and want more knobs: LM Studio.
FAQ
What model should I run on 16 GB of RAM? A Q4 quantization of a 7B or 8B model runs well in that envelope and covers most chat and coding use cases. Q4 quantization of a 13B is possible but tight.
Do these apps work on Apple Silicon? Yes. llama.cpp, Ollama, Jan, KoboldCpp, LM Studio, and Msty all use Metal on Apple Silicon and get near-native speed. llamafile ships with Apple Silicon binaries.
Can I point ChatGPT-style tools at a local model? Yes. Ollama, llama.cpp, Jan, KoboldCpp, LM Studio, and llamafile all expose an OpenAI-compatible endpoint on localhost. Point any tool that accepts a custom base URL at http://localhost:<port>/v1.
How is a minimalist stack different from a maximalist one? Fewer moving parts. One runtime, one model manager, one UI. Maximalist stacks add vector databases, orchestration layers, and agent frameworks. Most solo users do not need any of that until a specific problem demands it.
Are any of these agent-capable out of the box? No. These are inference runtimes and chat UIs. For an agent loop you add a separate tool that speaks the OpenAI Chat Completions protocol against one of these endpoints.
Which stack is the smallest on disk? llamafile plus a single quantized model is the lightest possible install. Ollama plus its model store is a close second and is easier to grow into.