跳到主要内容Skip to main content

梯度下降Gradient Descent

Gradient Descent

  • 核心概念Core concepts
  • 第四站 · 看懂Station 4 · Under the Hood
  • 引用核验 2026-11-17refs checked 2026-11-17
解决什么问题What problem it solves

沿着梯度的反方向小步更新参数,让损失逐步下降。Update parameters in the direction opposite the gradient, in small steps, so the loss goes down.

这是什么What it is

每次用一批数据算梯度,再按学习率缩放后更新权重。

SGD、Adam、AdamW 都是它的变体,差别在如何利用历史梯度。

Each step computes a gradient on a batch and scales it by the learning rate before updating weights.

SGD, Adam and AdamW are variants that differ in how they use gradient history.

Gradient descent takes repeated small steps on the loss surface: the gradient gives the direction and the learning rate gives the step size.
梯度下降在损失曲面上反复迈小步:方向由梯度给,步长由学习率给。Gradient descent takes repeated small steps on the loss surface: the gradient gives the direction and the learning rate gives the step size.
为什么需要知道Why it matters

学习率、批大小、优化器三者互相影响;同一套超参换模型或换数据常常失效。Learning rate, batch size and optimizer interact. The same hyperparameters often stop working when the model or data changes.

别名Chinese aliases

GD · SGD · 最速下降

出处Sources
  • PyTorch · 优化循环教程PyTorch optimization loop tutorial ↗
  • Goodfellow 等 · Deep Learning 第 8 章Goodfellow et al., Deep Learning, Chapter 8 ↗