投机解码Speculative Decoding
Speculative Decoding
解决什么问题What problem it solves
用小模型先猜几个词，大模型一次验证，生成更快。Guess several tokens cheaply, then have the big model verify them in one pass.
这是什么What it is
Leviathan 等人的定义：一种并行计算多个 token 的采样算法。前提是不改变输出。
原论文在 T5-XXL 上报告 2–3 倍加速，且输出完全一致。
vLLM 已内置 n-gram、EAGLE 等多种投机解码实现。
Leviathan et al.'s definition: an algorithm to sample from autoregressive models faster, computing several tokens in parallel with no changes to the outputs.
The paper reports 2x-3x acceleration on T5-XXL with identical outputs.
vLLM ships n-gram, EAGLE and other speculative decoding implementations.
为什么需要知道Why it matters
本地跑大模型时，这是少数「白拿」的提速手段之一。For local inference this is one of the few speedups you get for free.
别名Chinese aliases
推测解码