Quick answer
A local LLM is a language model that runs on your own computer instead of a company's servers. You download an open-weight model — Llama, Qwen, DeepSeek, Gemma, Mistral — with a free tool like Ollama or LM Studio, and it answers without an internet connection, without a subscription, and without your data leaving the machine. The trade-off is capability: a model that fits on a laptop is far weaker than the cloud frontier, and a model that competes needs serious hardware.
Every time you use ChatGPT, your words travel to a data centre, are processed on GPUs you will never see, and come back. A local LLM removes the journey. That single change is why developers, privacy-conscious professionals, and a growing number of ordinary users run models at home.
What you gain
- Privacy: nothing you type is sent anywhere, which matters for legal, medical, financial, and personal material
- Cost: zero per-token charges, no subscription, unlimited use
- Offline: works on a plane, in a secure facility, or when a provider has an outage
- Control: no vendor can change the model, the policy, or the price under you
- Learning: you see how models actually behave when the polish is removed
What you give up
- Capability: a 8B model on a laptop is roughly a 2023 frontier model; a 30B model on a workstation is roughly 2024
- Speed: once a model exceeds your GPU or unified memory, it slows to a crawl
- Convenience: you manage updates, storage, and model choice
- Features: no web browsing, image generation, or voice unless you add them yourself
The hardware question, answered simply
Any Apple silicon Mac with 16 GB of memory, or a Windows laptop with 16 GB of RAM, runs a compressed 7 to 9 billion parameter model well enough for drafting and summarising. 32 GB or a 24 GB GPU unlocks the 27 to 32 billion models that feel genuinely capable. Beyond that you are buying workstation GPUs. The tools tell you what fits before you download; you do not need to calculate anything.
Who should run one
- Developers: for coding help, testing, and building apps without an API bill
- Professionals with confidential material who cannot use cloud AI
- Anyone curious about how these systems work
- Probably not: people who just want the best answer to a hard question — the cloud still wins there, and hybrid setups like Perplexity's make the choice for you
Related reading
Bottom line
A local LLM trades the best possible answer for privacy, zero cost, and independence. In 2026 that trade is worth making for a growing share of everyday tasks, and the tools have made it a ten-minute setup rather than a weekend project.
