跳到主要内容Skip to main content

首字延迟Time To First Token, TTFT

Time To First Token, TTFT

  • 硬件与部署Hardware & deployment
  • 第五站 · 跑起来Station 5 · Running It
解决什么问题What problem it solves

从发出提问到看到第一个字要多久。这个数字决定「感觉快不快」。How long from sending a question to seeing the first word. This number decides whether it feels fast.

这是什么What it is

生成第一个 token 所花的时间。它主要由提示词长度和是否需要预填充决定。

The time taken to produce the first token, driven mainly by prompt length and whether prefill is needed.

为什么需要知道Why it matters

这是感知速度的关键指标。一个模型每字都很快但首字要等3 秒,用起来依然会觉得慢。The key metric for perceived speed. A model that generates fast per token but waits three seconds for the first one still feels slow.

出处Sources
  • vLLM 指标设计文档vLLM metrics design ↗
  • vLLM 性能基准套件vLLM benchmark suites ↗