通过一次评测就可以上线Passing One Evaluation Is Enough to Ship
微型指南 · 约 5 分钟Guide · about 5 min · 引用核验 2026-11-06(仅引用链接与日期) · refs checked 2026-11-06 (reference links and dates only)
上线不是通过一场考试,而是一个需要被持续检查的状态。Shipping is not passing one exam; it is a state that needs continuous checking.
上线前把评测跑通一次,之后就可以不用管了。Run the evaluation once before launch and you never have to look again.
那一次通过只覆盖当时的版本组合:模型版本、提示、检索到的材料、用户问法与数据分布,任何一项变动都可能让结论失效。发布后要留抽检、监控与回滚路径,改动时复跑同一套样本。(虚构示例)输入是一次「只更换检索源、模型与提示都没变」的改动 → 处理是复跑同一套样本 → 结果判断看的是「引用是否命中现行文档」这一项口径 → 若该项通过率下降,下一步是重新做上线判断并给检索源加一条变更检查,而不是沿用旧一次的「已通过」。That single pass covers only the version combination at the time: model version, prompt, retrieved material, user phrasing and data distribution - a change in any of them can void the result. After release keep sampling, monitoring and a rollback path, and rerun the same sample set on changes. (Fictional example) Input is a change that swaps only the retrieval source, with model and prompt untouched -> processing reruns the previous sample set -> the check looks at the single criterion "does the citation hit the current document" -> if that pass rate drops, the next step is to redo the go-live decision and add a retrieval-source change check, not to keep relying on last round's "passed".
例子与判断Example and interpretation
以下是教学情境,不是本站运行或实测记录。A teaching scenario, not a measured or executed site test.
把评测样本换成两周后的真实问法,同一套提示的通过率可能变化;上线门禁里要写明样本采集时间范围与多久复评一次,否则「通过」只描述过去。Swap the evaluation samples for how people actually ask two weeks later and the pass rate may move. Launch gates should state the collection window of the samples and how often they are refreshed, otherwise 'passed' only describes the past.
做完例子后怎么检查How to check your practice
先用自己的话解释上面的判断,再检查输入条件,最后回原文验证。说不清原因时回概念;缺日期、字段或权限时补输入;结论无出处时先撤回结论。Explain the example in your own words, inspect input constraints, then verify against the source. Revisit concepts if you cannot explain why; supply missing dates, fields or permissions; withdraw unsupported conclusions.
按这篇主题,先从“结果缺验证”练习查起For this topic, start with unchecked output · 查看全部六个情境与理由See all six scenarios and their reasoning
评估Evaluation, Evals 护栏Guardrails 持续集成Continuous Integration, CI
怎么让 AI 按格式输出How to Get AI to Output in Your Format · 怎么自建评测口径How to Build Your Own Eval · 提示词怎么版本化How to Version Your Prompts · 系统提示怎么设计How to Design a System Prompt