Core Concepts
Elo Rating (AI) in plain English.
Also known as: Arena Elo,LMArena score,chatbot arena rating
The one-sentence version
A ranking score for AI models derived from thousands of blind head-to-head votes, borrowed from chess ratings.
An Elo rating for an AI model is a score computed from many blind pairwise comparisons: a person asks a question, sees two anonymous answers, picks the better one, and each vote nudges the winner's score up and the loser's down, exactly as chess ratings work. LMArena (formerly Chatbot Arena) popularised this for chatbots, and specialised arenas now exist for coding, web development, and vision. Elo captures something benchmarks miss — which answers real people actually prefer — and it is hard to game by training on test questions. Its weaknesses are also real: voters reward confident, well-formatted, longer answers, so a model can climb by being pleasant rather than correct, and a gap of 20 points is close to noise. Treat Elo as one useful signal alongside task-specific benchmarks and your own testing.