学习率Learning Rate
Learning Rate
解决什么问题What problem it solves
控制每次参数更新的步长,是训练中最敏感的超参数之一。Set the step size of each parameter update; one of the most sensitive hyperparameters in training.
这是什么What it is
太大容易震荡或发散,太小收敛慢甚至卡在平台。
常见策略包括 warmup、余弦退火、按验证集表现衰减。
Too large causes oscillation or divergence; too small converges slowly or stalls on a plateau.
Common schedules include warmup, cosine decay and reducing on validation performance.
为什么需要知道Why it matters
换批大小、换优化器、做微调都要重新考虑学习率;微调通常比预训练小几个数量级。Changing batch size, optimizer or fine-tuning regime means reconsidering the learning rate. Fine-tuning usually uses rates orders of magnitude smaller than pretraining.
别名Chinese aliases
lr · 步长 · learning rate
同域词条Terms in the same area
现在能做什么What you can do now
回到名词地图看第4站的其他 46 个词 →Back to the map: the other 46 terms in station 4 →