论文卡-LIMAPaper Card - LIMA

研究问题：摘要原话——按两阶段训练 LIMA（65B 参数 LLaMa 模型），"fine-tuned with the standard supervised loss on only 1,000 carefully curated prompts and responses"（只用 1000 条精选问答做标准监督微调），"without any reinforcement learning or human preference modeling"（不做强化学习也不做人类偏好建模）——要测的就是两个训练阶段的相对重要性。

核心证据：第 4 节人工评测（对照 GPT-4/Bard/DaVinci003）与第 5 节消融（数据多样性/质量/数量）——摘要给出受控人评数字："responses from LIMA are either equivalent or strictly preferred to GPT-4 in 43% of cases"（43% 的情况下 LIMA 的回答与 GPT-4 相当或更好）；结论句是 "almost all knowledge in large language models is learned during pretraining"（几乎所有知识都在预训练阶段学到）。

作者自陈局限：没有独立的 Limitations 节（2026-10-30 核对 arXiv HTML 版）；第 7 节 Discussion 开头明确自陈两条——构造样例的心智成本难以规模化、"LIMA is not as robust as product-grade models"（不如产品级模型稳健，解码 unlucky sample 或对抗性提示常出弱回答）。

它没有主张的：43%/58%/65% 是「相当或更好」的受控人评占比，不是「整体达到 GPT-4 水平」；「1000 条足够」是对话风格层面的证据，论文没有主张安全、事实性、鲁棒性也能靠 1000 条解决——这三项恰恰是它自陈薄弱的地方。

读法提示：这篇的价值是把「知识来自预训练、对齐数据管的是风格」这个假设做成了可检验的消融；引用它时先分清「少数据能对齐」与「少数据能对齐一切」是两个命题。

Research question: the abstract describes training LIMA (a 65B LLaMa model) "fine-tuned with the standard supervised loss on only 1,000 carefully curated prompts and responses", "without any reinforcement learning or human preference modeling" — the point is to measure the relative importance of the two training stages.

Core evidence: Section 4's human evaluation (against GPT-4, Bard, DaVinci003) and Section 5's ablations on data diversity/quality/quantity — the abstract gives the controlled human-study number: "responses from LIMA are either equivalent or strictly preferred to GPT-4 in 43% of cases"; the conclusion line is "almost all knowledge in large language models is learned during pretraining".

Author-stated limits: no standalone Limitations section (checked against the arXiv HTML version on 2026-10-30); Section 7's Discussion states two limits outright — the mental effort of building such examples is hard to scale, and "LIMA is not as robust as product-grade models" (an unlucky decode or an adversarial prompt often yields a weak response).

What it does not claim: 43%/58%/65% are shares of "equivalent or strictly preferred" in a controlled human study — not "matches GPT-4 overall"; "1,000 examples suffice" is evidence about dialogue style, not about safety, factuality, or robustness — the very areas the authors admit are weak.

How to read it: its value is turning the hypothesis "knowledge comes from pretraining, alignment data shapes style" into testable ablations; when citing it, separate "little data can align a model" from "little data can align everything".