语义缓存Semantic cache
Semantic cache
解决什么问题What problem it solves
把语义相近的历史问题直接命中缓存答案,少调一次模型。Answer semantically similar questions from a cache and skip another model call.
这是什么What it is
把请求编码成向量,和缓存里的历史请求算相似度。
超过阈值就返回缓存答案;低于阈值才真正调用模型。
阈值决定「省多少钱」和「答错多少」之间的取舍。
Encode the request and compare its vector with cached historical requests.
Above a similarity threshold, return the cached answer; below it, call the model.
The threshold trades cost savings against wrong-answer risk.
为什么需要知道Why it matters
客服、FAQ 类流量里重复问题很多。语义缓存能显著降成本,但阈值太松会把不同问题当成同一个。FAQ and support traffic repeats heavily. Semantic caching cuts cost, but a loose threshold merges different questions.
同域词条Terms in the same area
现在能做什么What you can do now
回到名词地图看第3站的其他 66 个词 →Back to the map: the other 66 terms in station 3 →
出处Sources
- GPTCache 仓库GPTCache: semantic cache for LLM applications (GitHub) ↗