检索质量度量Retrieval Quality Metrics

常用口径是 recall@k：标准答案所在的那一段有没有出现在前 k 个结果里。Anthropic 讲检索的官方文章明说自己用 1 减 recall@20 作为失败率指标，也就是前 20 条里都没捞到正确片段的比例。

检索要分两步看：词面命中（BM25）与向量相似各自能捞回什么，再叠一层重排。Anthropic 报告把词面索引、上下文嵌入和重排叠起来以后，同一个指标上的失败率明显下降。那是他们内部题集上的结果，换一批文档就得重测，不能当成你的数字引用。

生成侧还要再看一次。ALCE 那篇论文把带引用的生成拆成流畅度、正确性、引用质量三类指标，其中引用质量看的是这句话能不能被它所引的来源支持；Foundry 的内置评估器里也有 Groundedness 与 Relevance 两项可以直接落到这一步。

你自己的文档上是多少，官方不会公布，属于未知。没有一份带标准答案的检索题集，任何关于检索质量的讨论都只是感觉。

The standard measure is recall@k — whether the chunk containing the answer appears in the top k results. Anthropic's own retrieval write-up says it uses one minus recall at 20 as the failure rate, meaning the share of cases where the right chunk was nowhere in the first twenty.

Look at retrieval in two parts: what lexical matching (BM25) returns, what vector similarity returns, and what a reranking stage does on top. Anthropic reports that combining a lexical index, contextual embeddings and reranking cut failures on that measure noticeably — on their internal set, so it is a method to copy, not a number to quote as yours.

The generation side needs a second look. The ALCE paper splits attributed generation into fluency, correctness and citation quality, where citation quality asks whether a sentence is actually supported by the source it cites; Foundry ships Groundedness and Relevance evaluators that do the equivalent job.

What recall looks like on your documents is unknown — nobody publishes it for you. Without a retrieval test set with known answers, every conversation about retrieval quality is a matter of impression.