跳到主要内容Skip to main content

语义缓存Semantic cache

Semantic cache

  • 硬件与部署Hardware & deployment
  • 第三站 · 用深Station 3 · Going Deeper
  • 引用核验 2026-11-19refs checked 2026-11-19
解决什么问题What problem it solves

把语义相近的历史问题直接命中缓存答案,少调一次模型。Answer semantically similar questions from a cache and skip another model call.

这是什么What it is

把请求编码成向量,和缓存里的历史请求算相似度。

超过阈值就返回缓存答案;低于阈值才真正调用模型。

阈值决定「省多少钱」和「答错多少」之间的取舍。

Encode the request and compare its vector with cached historical requests.

Above a similarity threshold, return the cached answer; below it, call the model.

The threshold trades cost savings against wrong-answer risk.

为什么需要知道Why it matters

客服、FAQ 类流量里重复问题很多。语义缓存能显著降成本,但阈值太松会把不同问题当成同一个。FAQ and support traffic repeats heavily. Semantic caching cuts cost, but a loose threshold merges different questions.

出处Sources
  • GPTCache 仓库GPTCache: semantic cache for LLM applications (GitHub) ↗