课堂-RAG 来源核对Lesson - Checking Sources in RAG Answers

能把一次回答拆成三段责任：检索是否命中、片段是否够用、生成是否超出片段。

能为每个断言找到对应片段与位置，找不到就标为无依据。

能说明分块会切断上下文，重排只改候选顺序、不新增事实。

能提出一个改进动作，并说明用什么口径验证它有没有效。

Split one answer into three responsibilities: whether retrieval hit, whether passages are sufficient, and whether generation goes beyond passages.

Find the passage and position for each claim; mark it unsupported if none can be found.

Explain that chunking can cut context and reranking only changes candidate order, not facts.

Propose one improvement and state how to verify whether it worked.

Anthropic 官方工程博客《Introducing Contextual Retrieval》：传统 RAG 编码时丢上下文会让检索失败。

Cohere 官方 Rerank 文档：给定查询与文档列表，按语义相关度排序。

LlamaIndex 官方 CitationQueryEngine 示例：答案带来源节点引用的写法。

三份虚构内部文档片段（其中一份条款被切断上下文）＋一张三列核对表。

Anthropic engineering post Introducing Contextual Retrieval: traditional RAG can lose context during encoding and fail retrieval.

Cohere Rerank documentation: given a query and document list, order by semantic relevance.

LlamaIndex CitationQueryEngine example: answers with source-node citations.

Three fictional internal document passages, one with a clause cut from its context, plus a three-column check sheet.

1. 3 分钟建参考答案：各组读问题与文档，先人工标出应当命中的片段。
2. 4 分钟查检索：对照系统给出的片段，分成命中、漏检、片段缺上下文三类。
3. 4 分钟查生成：逐句核对答案与片段，标出超出片段的断言与引用错位。
4. 4 分钟提改进：每组给一个动作与验证口径，全班按「改的是哪一段责任」归类。

1. 3-minute reference answer: groups read the question and documents and manually mark passages that should be hit.
2. 4-minute retrieval check: compare system passages and classify them as hit, missed, or missing context.
3. 4-minute generation check: check the answer sentence by sentence against passages and mark claims beyond passages or misplaced citations.
4. 4-minute improvement: each group gives one action and verification method; the class groups them by which responsibility stage changed.

以为答案带了引用就可信；引用可能指向不支持该句的片段。

把检索不到归因为模型笨；也可能是分块切断上下文或用词不一致。

以为重排能补出缺失事实；它只调整候选顺序。

把厂商博客里的提升幅度直接搬到自己的库；那是特定数据与设置下的结果。

Assuming an answer with citations is trustworthy; a citation may point to a passage that does not support the sentence.

Blaming the model for retrieval misses; chunking may cut context or wording may differ.

Assuming reranking can supply missing facts; it only adjusts candidate order.

Copying uplift figures from a vendor blog into your own corpus; those results depend on specific data and settings.

三列核对表填完：每个断言有片段位置，或明确标为无依据。

能把本组失败归因到三段中的一段，并指出支撑该判断的证据。

改进动作带验证口径，例如同一组问题重跑后比较无依据断言的数量。

The three-column sheet is complete: every claim has a passage position or is explicitly marked unsupported.

The group can attribute its failure to one of the three stages and point to evidence for that judgment.

The improvement has a verification method, such as rerunning the same questions and comparing the number of unsupported claims.