跳到主要内容Skip to main content

FrugalGPT(级联推理)FrugalGPT (LLM cascade)

FrugalGPT (LLM cascade)

  • 硬件与部署Hardware & deployment
  • 第三站 · 用深Station 3 · Going Deeper
  • 引用核验 2026-11-19refs checked 2026-11-19
解决什么问题What problem it solves

先用便宜的小模型试,置信度不够再升级到更强更贵的模型。Try a cheap small model first and escalate to a stronger, costlier model only when confidence is low.

这是什么What it is

请求先进入第一级模型,同时给出一个置信度或评分。

分数够高就直接返回;不够高才把同一请求转给下一级更大模型。

多级串起来形成级联,用平均成本换接近大模型的质量。

A request enters the first model, which also returns a confidence score.

A high score returns immediately; a low score sends the same request to the next, larger model.

A chain of levels trades average cost for quality close to the largest model.

为什么需要知道Why it matters

大多数请求其实不需要最贵的模型。级联能把成本压下来,但会引入延迟和路由判断错误。Most requests do not need the most expensive model. Cascades cut cost but add latency and router errors.

出处Sources
  • FrugalGPT 论文(arXiv)FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance (arXiv 2305.05176) ↗