跳到主要内容Skip to main content

模型合并Model Merging

Model Merging

  • 模型能力Model capabilities
  • 第四段 · 看懂Stage 4 · Under the Hood
  • 引用核验 2026-11-19refs checked 2026-11-19
解决什么问题What problem it solves

把两个或多个微调模型的权重合在一起,不重新训练就得到一个新的能力组合。Combine the weights of two or more fine-tuned models into a new capability mix without retraining.

这是什么What it is

常见做法包括线性加权、TIES、DARE 与任务向量;核心是处理参数冲突与冗余。

与集成(ensemble)的区别:合并产出一个模型,集成要同时跑多个模型。

合并前要确认同源底座与相同词表;不同底座或不同分词的模型直接合会坏。

许可证要逐个看:合并后的权重仍受各来源模型条款约束,不能因为「自己合的」就当成全新模型。

Common methods include linear weighting, TIES, DARE and task vectors; the core problem is parameter conflict and redundancy.

Unlike an ensemble, merging produces one model; an ensemble runs several models at once.

Check that the source models share a base and tokenizer; merging different bases or tokenizers breaks the result.

Read every licence: merged weights remain bound by each source model’s terms and are not a brand-new model just because you merged them.

为什么需要知道Why it matters

开源社区大量「合并模型」刷榜;不懂合并,容易把拼装当成新训练,也会误判许可证。Open-source communities merge models at scale to chase leaderboards; without understanding merging, repackaging looks like new training and licences get misjudged.

出处Sources
  • MergeKit 官方仓库MergeKit official repository ↗
  • Hugging Face · Merge models 指南Hugging Face · Merge models guide ↗
  • 学 AI · 微调Learn AI · Fine-tuning ↗