Benchmark
A standard test given to many models so their scores can be compared. Useful for rough rankings, easy to game, and never quite the same as how a model does on your actual work.
A benchmark is a fixed set of questions or tasks with known answers, used to score models against each other: grade-school math problems, coding puzzles, multiple-choice exams across dozens of subjects, long documents with a fact hidden inside. When a company announces a new model, the chart of benchmark scores is usually the first thing they show.
Benchmarks are useful because they’re the same for everyone. Their weakness is that a popular test becomes a target. Questions leak into the giant piles of text models train on, so a model can score well partly because it has effectively seen the answers, a kind of overfitting to the test. Labs also tune for the benchmarks they know they’ll be judged on. When a measure becomes a target, it stops being a good measure.
So scores are a rough guide, not a verdict. The test that matters most is the one nobody else can train on: your own tasks, in your own words.