HBM(高带宽显存)HBM (High Bandwidth Memory)
HBM (High Bandwidth Memory)
堆叠在 GPU 旁边的高带宽显存,决定大模型能不能装下、跑多快。High-bandwidth memory stacked next to the GPU; it decides whether a large model fits and how fast it decodes.
HBM 用硅通孔把多层 DRAM 垂直堆叠,再通过宽接口与 GPU 相连。
以 NVIDIA DGX B200 为例:8 张 GPU 共 1,440 GB HBM3e、合计 64 TB/s 带宽,折合单卡约 180 GB 与 8 TB/s。
HBM stacks DRAM layers with through-silicon vias and connects them to the GPU through a wide interface.
NVIDIA DGX B200 example: eight GPUs share 1,440 GB HBM3e and 64 TB/s aggregate bandwidth, about 180 GB and 8 TB/s per GPU.
显存容量决定能装多少参数和 KV cache,带宽决定解码速度;两者常是比算力更紧的瓶颈。Capacity sets how many parameters and KV cache fit; bandwidth sets decoding speed. Either can bind before raw compute does.
高带宽显存 · HBM3e · HBM 显存
回到名词地图看第5站的其他 38 个词 →Back to the map: the other 38 terms in station 5 →