模型合并Model Merging
Model Merging
把两个或多个微调模型的权重合在一起,不重新训练就得到一个新的能力组合。Combine the weights of two or more fine-tuned models into a new capability mix without retraining.
常见做法包括线性加权、TIES、DARE 与任务向量;核心是处理参数冲突与冗余。
与集成(ensemble)的区别:合并产出一个模型,集成要同时跑多个模型。
合并前要确认同源底座与相同词表;不同底座或不同分词的模型直接合会坏。
许可证要逐个看:合并后的权重仍受各来源模型条款约束,不能因为「自己合的」就当成全新模型。
Common methods include linear weighting, TIES, DARE and task vectors; the core problem is parameter conflict and redundancy.
Unlike an ensemble, merging produces one model; an ensemble runs several models at once.
Check that the source models share a base and tokenizer; merging different bases or tokenizers breaks the result.
Read every licence: merged weights remain bound by each source model’s terms and are not a brand-new model just because you merged them.
开源社区大量「合并模型」刷榜;不懂合并,容易把拼装当成新训练,也会误判许可证。Open-source communities merge models at scale to chase leaderboards; without understanding merging, repackaging looks like new training and licences get misjudged.
回到名词地图看第4段的其他 59 个词 →Back to the map: the other 59 terms in stage 4 →