怎么用 AI 搭 RAG 语料[object Object]
微型指南 · 约 8 分钟Guide · about 8 min · 引用核验 2026-11-19(仅引用链接与日期) · refs checked 2026-11-19 (reference links and dates only)
RAG 的效果八成取决于语料,而不是模型;先把「收什么、怎么切、怎么测」定下来。RAG quality is mostly corpus quality, not model quality. Decide what to collect, how to chunk and how to test first.
先定问题域和拒答边界:哪些问题必须答、哪些必须说不知道;语料只收能支撑这些问题的内容。
文档先去噪(页眉页脚、广告、导航),保留标题层级和来源 URL;扫描件先 OCR。
按语义结构切块,不按固定字数硬切;每块带文档标题、章节路径和来源链接。
给每块生成「可检索的问题/摘要」并保留原文,向量检索用问题、生成答案用原文。
建一个小评测集:真实问题 + 标准答案 + 必须引用的文档;每次换切法/模型都重跑。
定期清过期文档,保留版本与删除记录;不要让旧政策和新政策同时被检索到。
Define the question domain and refusal boundary: what must be answered, what must be “I don’t know”; only collect content that supports those.
Denoise documents (headers, footers, ads, navigation), keep heading hierarchy and source URL; OCR scans first.
Chunk by semantic structure, not fixed length; attach document title, section path and source link to each chunk.
Generate searchable questions/summaries per chunk but keep the original text: retrieve with questions, answer with originals.
Build a small eval set: real questions, gold answers, must-cite documents; rerun on every chunker or model change.
Retire stale documents with version and deletion records; never let old and new policies be retrieved together.
例子与判断Example and interpretation
以下是教学情境,不是本站运行或实测记录。A teaching scenario, not a measured or executed site test.
选 20 篇产品文档,去页眉页脚、按章节切块、每块带标题和 URL;用 10 个真实问题测命中与引用,记录哪类问题必须拒答。Take 20 product docs, strip headers/footers, chunk by section with title and URL, then test 10 real questions for hit rate and citation, recording which question types must be refused.
做完例子后怎么检查How to check your practice
先用自己的话解释上面的判断,再检查输入条件,最后回原文验证。说不清原因时回概念;缺日期、字段或权限时补输入;结论无出处时先撤回结论。Explain the example in your own words, inspect input constraints, then verify against the source. Revisit concepts if you cannot explain why; supply missing dates, fields or permissions; withdraw unsupported conclusions.
按这篇主题,先从“概念没分清”练习查起For this topic, start with concept confusion · 查看全部六个情境与理由See all six scenarios and their reasoning
分块Chunking 混合检索Hybrid retrieval 检索质量度量Retrieval Quality Metrics