XDA spent a week testing every small language model worth running on a normal laptop and came away with three keepers. The story landed the same week that Qwen 3, Llama 3.2, and Gemma 3 all released 1B–8B quants that punch far above their weight. The best apps for tiny local LLMs on desktop are no longer research toys. They fit in 4 GB of RAM, run without a discrete GPU, and answer everyday questions fast enough to replace the tab you keep open to ChatGPT.
We tested eight apps across an M2 MacBook Air (16 GB), a mid-range Ryzen laptop with iGPU only, and a Fedora workstation with a 6 GB NVIDIA card. Each was judged on how quickly a fresh install could pull a 3B quant, how well it handled a mixed workload of chat, code, and short document summaries, and whether it stayed usable while a video call was running in the background.
What to look for in a tiny local LLM app
- Small-model catalog. Direct pulls for Llama 3.2 (1B/3B), Qwen 3 (0.5B–8B), Gemma 3 (2B/4B), Phi-4-mini, and SmolLM without you hunting Hugging Face.
- Quantization presets. 4-bit and 5-bit GGUF quants are the sweet spot on 8–16 GB machines. The app should default to something sensible.
- CPU-only fallback. Not everyone has a GPU. The app should still be usable on integrated graphics or plain CPU.
- Memory footprint. How much RAM does the app itself take before you load a model. Some Electron shells cost 800 MB idle.
- OpenAI-compatible API. So a code editor, a note app, or a shell script can send prompts to the same process.
- Cold start. Time from double-click to a running chat matters when the reason you want it local is to skip the browser tab.
Quick comparison
| App | Best for | Platforms | Free plan | Paid | Rating |
|---|---|---|---|---|---|
| Jan | Open-source ChatGPT-shaped client | Windows, macOS, Linux | Fully free | None | 4.7 |
| Ollama | Simplest CLI + API runtime | Windows, macOS, Linux | Fully free | None | 4.9 |
| LM Studio | Model discovery with a real GUI | Windows, macOS, Linux | Fully free personal | Contact sales for work | 4.8 |
| GPT4All | Local + optional cloud fallback | Windows, macOS, Linux | Fully free | None | 4.5 |
| Msty | Split-chat comparison across models | Windows, macOS, Linux | Fully free | Aurum $99 lifetime | 4.7 |
| llamafile | Single-file portable model | Windows, macOS, Linux | Fully free | None | 4.6 |
| Cortex | Ollama-style runtime, lower footprint | Windows, macOS, Linux | Fully free | None | 4.4 |
| KoboldCpp | Long-context creative writing | Windows, macOS, Linux | Fully free | None | 4.5 |
The apps
1. Jan for tiny local LLMs — Best open-source ChatGPT-shaped client
Jan ships a ChatGPT-style desktop app that looks familiar the moment you open it, then quietly pulls Llama 3.2 3B or Qwen 3 4B in the background. Assistants, threads, file uploads, and an OpenAI-compatible server are built in. On the M2 Air, a first-time chat with Llama 3.2 3B was answering questions inside three minutes of install.
Where it falls short: Cold start on Windows still lags macOS by a few seconds. Some Hugging Face imports need a manual GGUF drop.
Pricing:
- Free: Fully free, open source under AGPL.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: jan.ai · github.com/janhq/jan
Bottom line: The pick if you want the ChatGPT experience without a subscription or an internet connection.
2. Ollama for tiny local LLMs — Best simplest CLI + API runtime
Ollama is the runtime everything else builds on. ollama run llama3.2:3b fetches the quant, loads it, and starts chatting. The OpenAI-compatible API on port 11434 means any editor, note app, or script that speaks OpenAI can talk to it without changes.
Where it falls short: No native GUI. You’ll pair it with Open WebUI or Msty for a chat window.
Pricing:
- Free: Fully free, MIT licensed.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: ollama.com · github.com/ollama/ollama
Bottom line: Start here even if you also install a chat client. Most other apps on this list can point at Ollama.
3. LM Studio for tiny local LLMs — Best model discovery with a real GUI
LM Studio is the all-in-one app for browsing Hugging Face, picking a quant, and chatting without touching a terminal. Its model browser filters by size and license, which matters when you’re trying to fit a model into 6 GB of RAM. The local API server is one toggle.
Where it falls short: Not open source. Linux support trails Windows and macOS on new features.
Pricing:
- Free: Fully free for personal use.
- Paid: Contact sales for commercial deployments.
Platforms: Windows, macOS, Linux.
Download: lmstudio.ai
Bottom line: The smoothest GUI for browsing and testing small quants side by side.
4. GPT4All for tiny local LLMs — Best local plus optional cloud fallback
GPT4All ships a native chat app that runs GGUF quants locally and can also route to OpenAI or Groq when a heavier model is needed. LocalDocs lets you point at a folder and get grounded answers without shipping files anywhere.
Where it falls short: Model catalog is a step behind LM Studio. GPU offload on Windows needs some knob-turning.
Pricing:
- Free: Fully free, open source.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: gpt4all.io
Bottom line: Pick this when you want one app that starts local and can borrow the cloud on demand.
5. Msty for tiny local LLMs — Best split-chat comparison across models
Msty shows two or three model responses side by side. That’s the fastest way to decide whether Qwen 3 4B or Gemma 3 4B fits your prompt style. It talks to Ollama, LM Studio, and cloud APIs, so the comparison isn’t limited to what it hosts.
Where it falls short: Some advanced features are behind the Aurum tier. Not open source.
Pricing:
- Free: Fully free chat and split view.
- Paid: Aurum $99 lifetime for knowledge stacks and premium extras.
Platforms: Windows, macOS, Linux.
Download: msty.app
Bottom line: The pick if you audition models often and want to see them argue.
6. llamafile for tiny local LLMs — Best single-file portable model
llamafile wraps a model and its runtime into one executable. Drop it onto a USB stick, double-click it on any Windows, macOS, or Linux machine, and a browser tab opens with a chat. It’s Mozilla’s contribution to Cosmopolitan libc, and it removes the install step entirely.
Where it falls short: Files can be 3–5 GB. First launch on locked-down corporate machines may need a right-click override.
Pricing:
- Free: Fully free, Apache 2.0.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: github.com/Mozilla-Ocho/llamafile
Bottom line: The right pick for portability, for demos, or for a machine you can’t install software on.
7. Cortex for tiny local LLMs — Best Ollama-style runtime with lower footprint
Cortex is the runtime that powers Jan under the hood, exposed as a standalone daemon. It’s a lighter footprint than Ollama at idle and defaults to llama.cpp with an OpenAI-compatible API. Model pulls use the same GGUF ecosystem.
Where it falls short: Smaller community, so third-party integrations are fewer. Documentation still catching up.
Pricing:
- Free: Fully free, open source.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: cortex.so · github.com/janhq/cortex.cpp
Bottom line: Pick this if Ollama feels heavy and you want a leaner daemon that still speaks the OpenAI API.
8. KoboldCpp for tiny local LLMs — Best for long-context creative writing
KoboldCpp is the fork of llama.cpp built around fiction, roleplay, and long-context writing. Persistent character memory, world info, and a browser UI focused on drafting rather than Q&A make it the odd one out on this list, in a useful way. It handles 32K–128K contexts on small models better than most GUIs.
Where it falls short: UI is dense and unashamedly power-user. Not the app to hand a non-technical friend.
Pricing:
- Free: Fully free, AGPL.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: github.com/LostRuins/koboldcpp
Bottom line: The pick if the reason you want a local model is writing sessions that ChatGPT would rate-limit.
How to pick the right one
If you want the simplest working setup: Ollama plus a chat client. Install Ollama, then pick Jan or Open WebUI on top.
If you want one polished desktop app: LM Studio for the GUI, or Jan if open source matters.
If you audition models often: Msty for the split view.
If you’re demoing to someone or working on a locked-down machine: llamafile.
If you want the lightest daemon: Cortex.
If you write long fiction or run roleplay campaigns: KoboldCpp.
If a tiny model isn’t enough for a specific task, most of these apps also route to cloud endpoints. Start local, escalate only when the local answer isn’t good enough.
FAQ
What is the best tiny local LLM app for a laptop with no GPU? Ollama with a 3B or 4B model runs on CPU and integrated graphics with acceptable speed. LM Studio ships CPU-only presets in its model picker.
Can a tiny local LLM replace ChatGPT? For everyday chat, summarization, and short-form code assistance, yes. For heavy reasoning, agentic workflows, and long research tasks, no. Most people who try both end up using local for daily prompts and cloud for the hard ones.
What size model runs on 8 GB of RAM? A 3B model in Q4 quantization fits comfortably with headroom for the app itself. A 4B quant works if you close other memory-heavy apps.
Are local LLMs actually private? The apps on this list don’t upload prompts by default. GPT4All and Msty offer optional cloud routing, and it’s opt-in. If total isolation matters, disable network access for the app.
What’s the best free tiny local LLM app? Ollama for the runtime and Jan for the GUI. Both are free, both are open source, both are actively maintained.
