FrugalGPT(级联推理)FrugalGPT (LLM cascade)
FrugalGPT (LLM cascade)
解决什么问题What problem it solves
先用便宜的小模型试,置信度不够再升级到更强更贵的模型。Try a cheap small model first and escalate to a stronger, costlier model only when confidence is low.
这是什么What it is
请求先进入第一级模型,同时给出一个置信度或评分。
分数够高就直接返回;不够高才把同一请求转给下一级更大模型。
多级串起来形成级联,用平均成本换接近大模型的质量。
A request enters the first model, which also returns a confidence score.
A high score returns immediately; a low score sends the same request to the next, larger model.
A chain of levels trades average cost for quality close to the largest model.
为什么需要知道Why it matters
大多数请求其实不需要最贵的模型。级联能把成本压下来,但会引入延迟和路由判断错误。Most requests do not need the most expensive model. Cascades cut cost but add latency and router errors.
同域词条Terms in the same area
现在能做什么What you can do now
回到名词地图看第3站的其他 66 个词 →Back to the map: the other 66 terms in station 3 →
出处Sources
- FrugalGPT 论文(arXiv)FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance (arXiv 2305.05176) ↗