JEPAJoint-Embedding Predictive Architecture
Joint-Embedding Predictive Architecture
让模型在表示空间里预测「被遮住的部分」,而不是在像素或 token 上逐字重建。Predict masked parts in representation space rather than reconstructing pixels or tokens.
输入和目标都先编码成表示,预测器在表示空间里预测目标表示。
它不要求重建原始像素 / 原始 token,因此可以把注意力放在可预测的语义结构上。
LeCun 2022 提出;2026 年的 JEPA-PRH 论文用它检验「跨模态表征收敛」假说。
Both input and target are encoded into representations; a predictor predicts the target representation.
It does not reconstruct raw pixels or tokens, so compute can focus on predictable semantic structure.
Proposed by LeCun in 2022; the 2026 JEPA-PRH paper uses it to test cross-modal representational convergence.
生成模型的主流是「重建下一个 token / 下一个像素」,JEPA 提供另一条路:先学世界如何变化,再决定要不要生成。理解它,才能读懂 2026 年关于「表征对齐」与「世界模型」的讨论,不会把「预测表示」误当成「生成质量」。Generative models mostly reconstruct the next token or pixel. JEPA offers another path: learn how the world changes first, then decide whether to generate. It is the context needed to read 2026 discussions of representation alignment and world models without mistaking “predicting representations” for “generation quality”.
回到名词地图看第6站的其他 36 个词 →Back to the map: the other 36 terms in station 6 →