自回归生成Autoregressive Generation

大模型「一个字一个字往外蹦」的生成方式，很多下游问题的根源。How large models emit text one token at a time — the root of many downstream issues.

自回归模型逐 token 生成。每一步以已生成的文本为条件，预测下一个 token。

GPT 系列论文对自己的定义就是「自回归语言模型」。

生成质量取决于每一步的选择，错误也会顺着往下带。

An autoregressive model generates token by token: each step predicts the next token conditioned on what has already been generated.

The GPT papers define their own models as autoregressive language models.

Output quality depends on every single step; errors get carried forward.

流式输出为什么是一个字一个字出来的？为什么中途能停？为什么一步选歪后面会顺着歪？根源都在这。Why streamed output arrives word by word, why it can be stopped mid-way, why one bad pick warps what follows — it all traces back to this.

去 Models 图谱：看这个模型有哪些量化版、按你的显存该下哪一档Models atlas: which quantisations exist and which tier fits your VRAM ↗

回到名词地图看第4站的其他 35 个词 →Back to the map: the other 35 terms in station 4 →

• GPT-3 论文（Brown et al. 2020，摘要自称 an autoregressive language model）GPT-3 paper (Brown et al. 2020) ↗