跳到主要内容Skip to main content

机械可解释性Mechanistic interpretability

Mechanistic interpretability

  • 核心概念Core concepts
  • 第四站 · 看懂Station 4 · Under the Hood
  • 引用核验 2026-11-19refs checked 2026-11-19
解决什么问题What problem it solves

把神经网络拆成可命名的特征与电路,解释它内部到底算了什么。Decompose a neural network into nameable features and circuits and explain what it computes internally.

这是什么What it is

目标不是看输入输出相关性,而是找到具体神经元/方向代表什么。

常用方法包括探针、激活 patching、稀疏自编码器和电路分析。

它试图回答「模型为什么给出这个输出」,而不只是「输出是什么」。

The goal is to identify what specific neurons or directions represent, not just input-output correlation.

Common tools include probes, activation patching, sparse autoencoders and circuit analysis.

It asks why the model produced an output, not only what the output is.

为什么需要知道Why it matters

安全审计、模型编辑和失败归因都需要知道内部机制;只看输入输出很难定位根因。Safety audits, model editing and failure attribution need internal mechanisms; input-output views rarely find root causes.

出处Sources
  • Towards Monosemanticity(Anthropic)Towards Monosemanticity: Decomposing Language Models With Dictionary Learning (Anthropic) ↗