Core Concepts
Local LLM in plain English.
Also known as: on-device LLM,running AI locally,local model
The one-sentence version
A language model that runs entirely on your own computer or server instead of a cloud provider's, so no data leaves your machine.
A local LLM is a language model you run on hardware you control — a laptop, a workstation, or your own server — rather than calling a cloud API. Open-weight models from Meta, Alibaba, DeepSeek, Google, and Mistral can be downloaded and run with tools like Ollama and LM Studio, often in compressed (quantised) form so they fit in ordinary memory. The appeal is privacy, zero per-token cost, offline use, and independence from any vendor's pricing or policy changes. The trade-off is capability: a model small enough to run on a laptop is far weaker than a frontier cloud model, and a model large enough to compete needs expensive GPUs. Hybrid setups, where routine tasks run locally and hard ones go to the cloud, are becoming the norm for developers and privacy-sensitive teams.