连续批处理Continuous Batching
Continuous Batching
解决什么问题What problem it solves
让服务器把不同时刻到达的请求拼在一起算，而不是排队。The server batches requests that arrived at different moments instead of queueing them.
这是什么What it is
vLLM 官方把它列为核心加速手段：对到达的请求做连续批处理。
官方把它与分块预填充、前缀缓存一起列为让 vLLM 快的手段。
vLLM lists it as a core acceleration technique: continuous batching of incoming requests.
Together with chunked prefill and prefix caching it forms the throughput base of a serving engine.
为什么需要知道Why it matters
同一个模型，服务引擎不同，能扛的并发差好几倍。Same model, different serving engine: concurrency differs by multiples.