Quantization (AI) in plain English.

Quantization is the process of shrinking an AI model by reducing the numerical precision used to store its weights — for example, converting weights stored as 16-bit numbers down to 8-bit or 4-bit versions. This cuts the model's memory footprint and speeds up inference substantially, often by half or more, at the cost of a small, usually acceptable drop in output quality. Quantization is why a chatbot with tens of billions of parameters can run on a high-end laptop or phone instead of requiring a data-center GPU: a quantized version of Llama 4 or Mistral can fit on consumer hardware where the full-precision original could not. Most on-device AI and locally-hosted open models in 2026 run quantized by default. The trade-off matters more on tasks requiring precise reasoning or exact numbers than on casual chat, where the quality difference is barely noticeable.