基准测试Benchmark

用统一的题库横向比较模型强弱。Compare models on a common question set.

标准化评测集（如 MMLU、HumanEval、SWE-bench、RULER 等）给出的分数。

不同基准测不同能力：知识问答、代码修复、长上下文、Agent 任务不能互相替代。

A score from a standardised evaluation set such as MMLU, HumanEval, SWE-bench or RULER.

Different benchmarks measure different abilities: knowledge Q&A, code repair, long context and agent tasks are not interchangeable.

重要但不可尽信——公开榜单容易过时，也可能与你的实际任务无关；看版本、训练截止日与是否去污染。Important but not gospel: public leaderboards age quickly and may not match your task; check the version, training cutoff and decontamination.

去 Models 图谱：看这个模型有哪些量化版、按你的显存该下哪一档Models atlas: which quantisations exist and which tier fits your VRAM ↗

回到名词地图看第6站的其他 29 个词 →Back to the map: the other 29 terms in station 6 →

• MMLU 论文Measuring Massive Multitask Language Understanding (MMLU) ↗
• SWE-bench 论文（arXiv 2310.06770）SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ↗
• RULER 论文（arXiv 2404.06654）RULER: What's the Real Context Size of Your Long-Context Language Models? ↗
• HELM 评测榜Holistic Evaluation of Language Models (HELM) ↗