怎么给文档做分块How to Chunk Documents for Retrieval
微型指南 · 约 7 分钟Guide · about 7 min · 引用核验 2026-11-03(仅引用链接与日期) · refs checked 2026-11-03 (reference links and dates only)
检索增强第一步是把文档切成块。块切得好不好,直接决定后面找不找得到答案。The first step of retrieval-augmented generation is cutting documents into chunks. How well you cut decides whether answers can be found at all.
按语义切,不按字数切:一个完整论点别被拦腰切断。
块太小:上下文丢失,检索命中也答不全。
块太大:一个块混多个主题,相似度被稀释。
常见起点是几百到一千字,再用效果调,不迷信固定值。
块之间留重叠:相邻块共享几句话,防止边界处信息断裂。
保留结构信息:标题、章节、页码存进元数据,检索时能过滤。
Cut by meaning, not by character count: never slice a complete argument in half.
Chunks too small: context is lost, and even a hit cannot support a full answer.
Chunks too large: one chunk mixes several topics, and the similarity signal gets diluted.
A few hundred to a thousand characters is a common starting point; tune by results, do not trust a fixed value.
Overlap adjacent chunks: sharing a few sentences prevents information from breaking at the boundary.
Keep structural information: store titles, sections, and page numbers as metadata so retrieval can filter on them.
例子与判断Example and interpretation
以下是教学情境,不是本站运行或实测记录。A teaching scenario, not a measured or executed site test.
一份退货说明把“七天内可退”和“定制品除外”分在相邻两段。若只命中前一句,就可能漏掉限制。尝试把规则和例外放进同一信息单元,再用“定制品能退吗”检查检索结果。A returns policy puts “within seven days” and “except customized items” in adjacent paragraphs. Retrieving only the first can omit the exception. Keep the rule and exception together, then test retrieval for customized items.
分块Chunking 联网 / 检索增强Retrieval-Augmented Generation, RAG 向量 / 嵌入Embedding / Vector
怎么给检索结果重排How to Rerank Retrieval Results · 怎么做混合检索How to Do Hybrid Search · 怎么挑 embedding 模型How to Choose an Embedding Model · 怎么评测 RAG 的效果How to Evaluate a RAG System