跳到主要内容Skip to main content

多卡互联值不值Is Multi-GPU Interconnect Worth It?

微型指南 · 约 8 分钟Guide · about 8 min · 引用核验 2026-11-19(仅引用链接与日期) · refs checked 2026-11-19 (reference links and dates only)

多卡不是「显存加法」;先算通信、并行方式与真实吞吐,再决定掏不掏钱。More GPUs is not additive VRAM. Calculate communication, parallelism and real throughput before spending.

怎么做How to

先判断瓶颈:单卡显存不够、单卡算力不够、还是要更大并发;三种问题解法不同。

张量并行吃互联带宽,流水并行吃显存与气泡;NVLink、PCIe、雷电的差距先查清。

算有效吞吐:不是峰值 FLOPS 相加,而是端到端 tokens/s 与首字延迟。

算总拥有成本:卡、主板、电源、散热、机房、电费、运维;别只算卡价。

小模型与低并发通常单卡更划算;多卡适合大模型、长上下文、高并发。

先租后买:用云上同规格多卡跑一周真实负载,再决定。

Find the bottleneck first: not enough VRAM, not enough compute, or need more concurrency. Each has a different fix.

Tensor parallelism eats interconnect bandwidth; pipeline parallelism eats VRAM and bubbles. Check NVLink, PCIe and Thunderbolt gaps first.

Calculate effective throughput: not summed peak FLOPS, but end-to-end tokens/s and time to first token.

Calculate total cost of ownership: cards, board, PSU, cooling, room, power, operations; not just card price.

Small models at low concurrency are usually cheaper on one card; multi-GPU suits large models, long context and high concurrency.

Rent before buying: run the real workload for a week on the same cloud configuration, then decide.

例子与判断Example and interpretation

以下是教学情境,不是本站运行或实测记录。A teaching scenario, not a measured or executed site test.

同一台机器先跑单卡 70B 4bit 的 tokens/s 与首字延迟,再跑双卡张量并行;如果双卡只提升 20% 而功耗翻倍,就先别买第二张卡,改用更小模型或量化。Benchmark a 70B 4-bit model on one GPU for tokens/s and time-to-first-token, then on two GPUs with tensor parallelism. If two cards add only 20% throughput while doubling power, do not buy the second card; use a smaller model or stronger quantization.

做完例子后怎么检查How to check your practice

先用自己的话解释上面的判断,再检查输入条件,最后回原文验证。说不清原因时回概念;缺日期、字段或权限时补输入;结论无出处时先撤回结论。Explain the example in your own words, inspect input constraints, then verify against the source. Revisit concepts if you cannot explain why; supply missing dates, fields or permissions; withdraw unsupported conclusions.

按这篇主题,先从“概念没分清”练习查起For this topic, start with concept confusion · 查看全部六个情境与理由See all six scenarios and their reasoning

出处Sources
  • NVIDIA · NVLinkNVIDIA · NVLink ↗
  • vLLM · Distributed servingvLLM · Distributed serving ↗
  • 算力与硬件站 · 硬件算力Hardware site · Hardware compute ↗
围绕这些概念继续读More on these concepts

接口兼容就等于行为一致A Compatible API Means Identical Behaviour