数据污染Data Contamination
Data Contamination
评测题混进训练数据，模型「考过原题」——评测分数因此失真。Eval questions leaking into training data — models have 'seen the exam', so scores become distorted.
指评测集的内容在预训练阶段混进了训练数据，被模型提前见过。
GPT-3 论文自己就做过 n-gram 重叠检查，评估各评测集被污染的影响。
Magar 与 Schwartz 进一步把污染分成两种行为：「记住」与「利用」。该研究对两者做了量化。
静态评测集公开可得，污染几乎不可避免。
Contamination means evaluation-set content was inadvertently seen during pre-training.
The GPT-3 paper itself ran n-gram overlap checks to assess contamination across eval sets.
Magar & Schwartz split contamination into memorization and exploitation, and quantified both.
With static eval sets publicly available, contamination is nearly unavoidable.
榜单上「某模型在 X 基准特别高」，可能只是它见过原题。评测评判模型前先问：这个基准会不会已经漏进训练数据。'Model X tops benchmark Y' may just mean it saw the questions. Before trusting a leaderboard slot, ask whether that benchmark has already leaked into training data.
回到名词地图看第4站的其他 35 个词 →Back to the map: the other 35 terms in station 4 →