分块预填充Chunked Prefill
Chunked Prefill
解决什么问题What problem it solves
把很长的输入切成小块处理，顺便和生成中的请求混在一起跑。Process a long input in smaller chunks, mixing it with requests that are generating.
这是什么What it is
vLLM 官方定义：允许把大的预填充拆成较小的块，并与解码请求一起批处理。
官方称它通过平衡计算密集与显存密集两类操作，同时改善吞吐和延迟。
vLLM's definition: process large prefills in smaller chunks and batch them with decode requests.
The docs say it improves both throughput and latency by balancing compute-bound and memory-bound work.
为什么需要知道Why it matters
长输入把首字延迟拖爆时，这是服务端最常用的缓解手段。When a long input wrecks time-to-first-token, this is the standard server-side mitigation.
现在能做什么What you can do now
回到名词地图看第5站的其他 29 个词 →Back to the map: the other 29 terms in station 5 →
出处Sources
- vLLM 优化配置文档vLLM optimization configuration ↗