稀疏自编码器Sparse autoencoder
Sparse autoencoder
解决什么问题What problem it solves
把模型内部密密麻麻的激活拆成一组稀疏、可命名的特征方向。Split dense model activations into a sparse set of nameable feature directions.
这是什么What it is
在模型某层的激活上训练一个宽得多的自编码器。
加稀疏惩罚,让每个输入只激活少量隐藏单元。
这些隐藏单元就是候选的「单义特征」,比原始神经元更容易解释。
Train a much wider autoencoder on activations from one layer.
Add a sparsity penalty so each input activates only a few hidden units.
Those units become candidate monosemantic features that are easier to interpret.
为什么需要知道Why it matters
原始神经元经常一个神经元管多个概念(多义性)。稀疏自编码器是可解释性研究的主流工具。Raw neurons are often polysemantic. Sparse autoencoders are a mainstream interpretability tool.
同域词条Terms in the same area
现在能做什么What you can do now
回到名词地图看第4站的其他 54 个词 →Back to the map: the other 54 terms in station 4 →
出处Sources
- Towards Monosemanticity(Anthropic)Towards Monosemanticity: Decomposing Language Models With Dictionary Learning (Anthropic) ↗