专家卸载MoE Offload / ncmoe

对「专家很多但每次只激活一部分」的稀疏模型，把不常用的专家放内存里。For sparse models with many experts where only a few activate per token, keep the rarely-used experts in memory.

混合专家（MoE）模型每次推理只激活少数专家，其余权重可放在较慢的存储上按需调取。

A mixture-of-experts (MoE) model activates only a few experts per inference, so the remaining weights can sit on slower storage and be fetched as needed.

它解释了为什么有些超大模型在单卡上也能跑。因为不是所有权重都同时参与计算。It explains why some very large models run on a single card: not all weights participate at once.

去 Models 图谱：看这个模型有哪些量化版、按你的显存该下哪一档Models atlas: which quantisations exist and which tier fits your VRAM ↗

回到名词地图看第5站的其他 29 个词 →Back to the map: the other 29 terms in station 5 →

• llama.cpp CLI 参数文档llama.cpp CLI arguments ↗