机械可解释性Mechanistic interpretability
Mechanistic interpretability
把神经网络拆成可命名的特征与电路,解释它内部到底算了什么。Decompose a neural network into nameable features and circuits and explain what it computes internally.
目标不是看输入输出相关性,而是找到具体神经元/方向代表什么。
常用方法包括探针、激活 patching、稀疏自编码器和电路分析。
它试图回答「模型为什么给出这个输出」,而不只是「输出是什么」。
The goal is to identify what specific neurons or directions represent, not just input-output correlation.
Common tools include probes, activation patching, sparse autoencoders and circuit analysis.
It asks why the model produced an output, not only what the output is.
安全审计、模型编辑和失败归因都需要知道内部机制;只看输入输出很难定位根因。Safety audits, model editing and failure attribution need internal mechanisms; input-output views rarely find root causes.
看站群怎么分工:这一站回答「是什么」,其他站回答「选哪个」How the site family divides the work ↗
回到名词地图看第4站的其他 54 个词 →Back to the map: the other 54 terms in station 4 →
- Towards Monosemanticity(Anthropic)Towards Monosemanticity: Decomposing Language Models With Dictionary Learning (Anthropic) ↗