SWE-bench in plain English.

SWE-bench is a benchmark built from real GitHub issues in popular open-source Python projects. For each task the model is given the repository and the issue text and must produce a code change; it passes only if the project's hidden tests, which previously failed, now succeed. SWE-bench Verified is a human-checked subset of 500 tasks that removed ambiguous or broken problems, and it became the headline number every lab quotes for coding agents. Scores climbed from under 5% in 2023 to above 80% by 2026, which says as much about the benchmark saturating as about progress, and it has spawned successors — Terminal-Bench, FrontierCode, CursorBench — that test longer, messier, multi-language work. Useful rules: compare models on the same version, be suspicious of scores that arrive with no test harness details, and remember that fixing a well-specified Python issue is not the same as building a product.