怎么评测 RAG 的效果How to Evaluate a RAG System
微型指南 · 约 8 分钟Guide · about 8 min · 引用核验 2026-11-03（仅引用链接与日期） · refs checked 2026-11-03 (reference links and dates only)
RAG 出问题可能出在检索，也可能出在生成。评测要分开量，不然修错地方。A RAG failure can come from retrieval or from generation. Evaluate them separately, or you will fix the wrong part.
先备评测集：真实问题加标准答案，几十条就够起步。
检索侧量召回：该找到的块有没有进候选，这是第一道关。
生成侧量忠实度：回答里的每句话能不能被检索到的块支撑。
答案相关性单独量：答得对题但依据错，也是问题。
用 RAGAS 这类框架自动评，人工抽查校准它的判分。
分场景看分数：不同问题类型分开统计，平均分掩盖短板。
Build an eval set first: real questions with gold answers; a few dozen is enough to start.
Retrieval side, measure recall: did the chunk that should be found make it into the candidates?
Generation side, measure faithfulness: can every sentence in the answer be supported by the retrieved chunks?
Measure answer relevance separately: on-topic answers with wrong grounding are still a problem.
Automate with a framework like RAGAS, then spot-check by hand to calibrate its scoring.
Break scores down by scenario: average scores hide the weak spots of specific question types.
例子与判断Example and interpretation
以下是教学情境，不是本站运行或实测记录。A teaching scenario, not a measured or executed site test.
问题问“报销上限”，正确文档已取回，回答却引用了餐费额度。这是生成或证据使用错误。若取回的全是旧版政策，则先修检索。把这两类失败分别记账，才知道该改哪一段。If the right reimbursement policy was retrieved but the answer uses a meal allowance, inspect generation/evidence use. If every retrieved document is outdated, inspect retrieval first. Record the two failure types separately.
联网 / 检索增强Retrieval-Augmented Generation, RAG 模型评测的局限Evaluation Limits 基准测试Benchmark
AI 常见误区总索引AI Misconceptions: Index · 怎么判断 AI 新闻是不是炒作How to Tell if AI News Is Hype · 怎么给文档做分块How to Chunk Documents for Retrieval · 怎么给检索结果重排How to Rerank Retrieval Results