跳到主要内容Skip to main content

交叉熵Cross-Entropy

Cross-Entropy

  • 核心概念Core concepts
  • 第四站 · 看懂Station 4 · Under the Hood
  • 引用核验 2026-11-17refs checked 2026-11-17
解决什么问题What problem it solves

衡量预测分布与真实分布差多远,是分类与语言模型最常用的损失。Measure how far the predicted distribution is from the true one; the default loss for classification and language models.

这是什么What it is

交叉熵等于真实分布的负对数似然:给正确答案的概率越低,惩罚越大。

PyTorch 的 CrossEntropyLoss 把 log-softmax 与负对数似然合在一起。

Cross-entropy is the negative log-likelihood of the true distribution: the lower the probability given to the right answer, the larger the penalty.

PyTorch’s CrossEntropyLoss combines log-softmax and negative log-likelihood.

Cross-entropy looks at the probability assigned to the correct class: near 1 gives near-zero loss, near 0 gives a large penalty.
交叉熵只看正确答案那一项的概率:概率越接近 1,损失越接近 0;越接近 0,惩罚越大。Cross-entropy looks at the probability assigned to the correct class: near 1 gives near-zero loss, near 0 gives a large penalty.
为什么需要知道Why it matters

语言模型的困惑度就是交叉熵的指数形式;看到 loss 数值要先确认单位与词表口径。Perplexity is the exponential of cross-entropy. Before comparing loss numbers, check the unit and the vocabulary denominator.

别名Chinese aliases

CE · cross entropy · 负对数似然

出处Sources
  • PyTorch · CrossEntropyLossPyTorch CrossEntropyLoss ↗
  • Goodfellow 等 · Deep Learning 第 6 章Goodfellow et al., Deep Learning, Chapter 6 ↗