怎么做跨模型的一致性回归测试How to Run Cross-Model Regression Tests
微型指南 · 约 9 分钟Guide · about 9 min · 引用核验 2026-11-17(仅引用链接与日期) · refs checked 2026-11-17 (reference links and dates only)
模型升级不是「换个更强的就行」;同一套回归集必须前后跑,才能知道哪些能力变了、哪些只是风格变了。A model upgrade is not just “a stronger model”. The same regression set must run before and after to separate capability changes from style changes.
先固定 50–100 条回归集,覆盖常见任务、边界输入和必须拒答的场景。
每条写期望要点而不是唯一答案,用要点覆盖率打分。
升级前后跑同一套集,记录通过率、引用正确率、拒答率和平均长度。
分开统计能力变化与风格变化:变长、变客气不等于变好。
模型评审只做初筛,抽 20% 人工复核,防止位置与冗长偏差。
设回滚阈值:低于阈值就回滚,不要用「整体感觉更好」上线。
保留每次原始输出;系统提示、工具描述、检索库更新也各算一次变更。
Fix a 50–100 case regression set covering common tasks, edge inputs and must-refuse scenarios.
Write expected bullet points rather than one gold answer, and score coverage of those points.
Run the same set before and after, recording pass rate, citation accuracy, refusal rate and average length.
Report capability change and style change separately: longer or more polite is not better.
Use model judging only for triage, with 20 percent human review to counter position and verbosity bias.
Set a rollback threshold and use it; do not ship on “it feels better overall”.
Keep every raw output, and count system-prompt, tool-description and retrieval-index changes as changes too.
例子与判断Example and interpretation
以下是教学情境,不是本站运行或实测记录。A teaching scenario, not a measured or executed site test.
建一个 60 条的回归集,其中 20 条是「必须拒答或必须说不知道」;模型升级时,这 20 条的通过率比总分更重要。Build a 60-case regression set, 20 of which must be refused or answered with “I don’t know”. On a model upgrade, the pass rate on those 20 matters more than the total score.
做完例子后怎么检查How to check your practice
先用自己的话解释上面的判断,再检查输入条件,最后回原文验证。说不清原因时回概念;缺日期、字段或权限时补输入;结论无出处时先撤回结论。Explain the example in your own words, inspect input constraints, then verify against the source. Revisit concepts if you cannot explain why; supply missing dates, fields or permissions; withdraw unsupported conclusions.
按这篇主题,先从“概念没分清”练习查起For this topic, start with concept confusion · 查看全部六个情境与理由See all six scenarios and their reasoning
离线评测与在线评测Offline vs Online Evaluation 评测集设计Eval Set Design 模型评测的局限Evaluation Limits
怎么评测 RAG 的效果How to Evaluate a RAG System · 怎么自建评测口径How to Build Your Own Eval · 榜单怎么读How to Read a Leaderboard · 怎么读一篇AI论文How to Read an AI Paper