GPU 层卸载 · 学 AI

解决什么问题What problem it solves

把模型一部分交给 GPU、一部分留在 CPU，显存不够时的妥协手段。Give part of the model to the GPU and keep the rest on the CPU — the compromise when VRAM falls short.

这是什么What it is

按层分配计算位置：显存够的部分放 GPU，剩下的放 CPU 由内存承担。

llama.cpp 里通常用 `-ngl` 参数控制。

Assigning layers across devices: whatever fits in VRAM goes to the GPU, the rest runs on the CPU backed by system memory.

In llama.cpp this is usually controlled with the `-ngl` flag.

为什么需要知道Why it matters

它让「显存差一点」的模型也能跑起来，代价是速度下降明显。这是权衡，不是免费午餐。It lets models that "almost" fit run at all, at a clear cost in speed. A trade-off, not a free lunch.

接着读什么，为什么What to read next, and why

CPU 推理CPU Inference
一起用于《显存不够就跑不了》Used together in “Not Enough VRAM Means It Will Not Run”

现在能做什么What you can do now

出处Sources

GPU 层卸载-ngl / Offload