跳到主要内容Skip to main content

视频生成是怎么做的How Video Generation Works

How Video Generation Works

  • 图像语音视频Image, speech & video
  • 第四站 · 看懂Station 4 · Under the Hood
解决什么问题What problem it solves

视频生成不是「逐帧生图」那么简单:时间一致性才是难点。Video generation is not simply 'generate one frame at a time': temporal consistency is the hard part.

这是什么What it is

视频扩散模型在图像扩散的基础上加入时间维度,让相邻帧保持连贯。

《Video Diffusion Models》(Ho 等,2022)是早期代表作。先建空间模型,再加时序层。

难点在一致性:物体移动、镜头运动都要跨帧连贯,模型常在这里出错。

时长与一致性是当前主要限制。这也是它比生图更不成熟的原因。

Video diffusion models add a temporal dimension on top of image diffusion so neighbouring frames stay coherent.

Video Diffusion Models (Ho et al., 2022) is an early landmark. Build the spatial model first, then add temporal layers.

The difficulty is consistency — object motion and camera motion must stay coherent across frames, and models fail here most often.

Duration and consistency are the current limits. That is why it is less mature than image generation.

为什么需要知道Why it matters

选视频工具时,能生成多长、稳不稳,比画面好不好看更能区分产品。When choosing a video tool, 'how long can it generate, how stable is it' separates products better than 'does the picture look good'.

出处Sources
  • 《Video Diffusion Models》论文(arXiv)Video Diffusion Models (arXiv 2204.03458) ↗
  • OpenAI Sora 官方页OpenAI Sora (official) ↗