交叉熵Cross-Entropy
Cross-Entropy
解决什么问题What problem it solves
衡量预测分布与真实分布差多远,是分类与语言模型最常用的损失。Measure how far the predicted distribution is from the true one; the default loss for classification and language models.
这是什么What it is
交叉熵等于真实分布的负对数似然:给正确答案的概率越低,惩罚越大。
PyTorch 的 CrossEntropyLoss 把 log-softmax 与负对数似然合在一起。
Cross-entropy is the negative log-likelihood of the true distribution: the lower the probability given to the right answer, the larger the penalty.
PyTorch’s CrossEntropyLoss combines log-softmax and negative log-likelihood.
为什么需要知道Why it matters
语言模型的困惑度就是交叉熵的指数形式;看到 loss 数值要先确认单位与词表口径。Perplexity is the exponential of cross-entropy. Before comparing loss numbers, check the unit and the vocabulary denominator.
别名Chinese aliases
CE · cross entropy · 负对数似然
同域词条Terms in the same area
现在能做什么What you can do now
回到名词地图看第4站的其他 46 个词 →Back to the map: the other 46 terms in station 4 →