幻觉检测信号Hallucination Warning Signs
Hallucination Warning Signs
在没有标准答案的场合，靠哪几个信号判断这段回答可能在编。Which signals tell you a passage is probably invented when you have no answer key to check against.
最省事的信号来自反复问。SelfCheckGPT 的做法是对同一个问题采样多次，如果几次回答里的事实互相对不上，这段内容更可能是编的。这个方法不需要模型内部的概率，黑箱接口也能用。
第二个信号是可核对性：回答里给出的日期、条款号、文献题名、接口名能不能被一条命令验证。编造最容易出现在具体编号上，而不是在概括里。
第三方检测也在往这个方向做。微软内容安全里有 groundedness detection，检查回答是否落在你提供的材料上；Foundry 的内置评估器里同样有 Groundedness 一项。要注意它测的是「有没有依据」，不等于「说得对不对」。
模型自己说的把握不算证据。它写「我确定」和写「大概是」，与这段内容是否真实没有稳定关系；校准是另一件事。
一个虚构示例：同一道「列出去年的三项版本号」的问题，第一次答 A1.2、第二次答 A2.0、第三次开始道歉说记不准。前后不一致本身就是该去查原始页面的信号。
The cheapest signal comes from asking twice. SelfCheckGPT samples several answers to the same question and flags the ones whose facts disagree with each other. It needs no internal probabilities, so it works against a black-box API.
The second signal is checkability: can the dates, clause numbers, paper titles and endpoint names in the answer be verified with one command. Fabrication concentrates in specific identifiers rather than in general summaries.
Third-party checks are heading the same way. Microsoft's content-safety service includes groundedness detection, which asks whether a response stays within the material you supplied, and Foundry ships a Groundedness evaluator. Note what it measures is support, not truth.
A model's own confidence is not evidence. Writing I am sure and writing roughly or probably have no stable relationship to whether the content is real; calibration is a separate problem.
A fictional worked example: asked once for three version numbers from last year the answer says A1.2, asked again it says A2.0, asked a third time it apologises for being unsure. The disagreement itself is the signal to go check the source page.
你不可能把每句话都送去人工核对。知道信号在哪，才能把有限的核对时间花在编号、引用和结论上，而不是通读一遍还看不出问题。Nobody verifies every sentence by hand. Knowing where the signals sit lets you spend scarce checking time on identifiers, citations and conclusions instead of rereading the whole answer and still missing the problem.
回到名词地图看第2站的其他 41 个词 →Back to the map: the other 41 terms in station 2 →