张量并行Tensor Parallelism
Tensor Parallelism
解决什么问题What problem it solves
把一个模型拆到多张卡上一起算，单卡装不下时用。Split one model across several GPUs when it does not fit on a single card.
这是什么What it is
把模型权重按维度切分到多张 GPU，每张算一部分，需要卡间通信。
Partitioning model weights across multiple GPUs by dimension, each computing part, with inter-card communication required.
为什么需要知道Why it matters
多卡部署的主要手段。显存够就优先单卡——通信开销会拖慢速度。The main technique for multi-GPU deployment. Prefer a single card when VRAM allows — communication overhead slows things down.
别名Chinese aliases
TP