跳到主要内容Skip to main content

多模态推理Multimodal Reasoning

Multimodal Reasoning

  • 模型能力Model capabilities
  • 第四段 · 看懂Stage 4 · Under the Hood
  • 引用核验 2026-11-19refs checked 2026-11-19
解决什么问题What problem it solves

让模型在看图或看视频之后还能多步推理,而不是只做一次描述。Let a model reason over several steps after seeing an image or video, not just describe it once.

这是什么What it is

输入不只是文本,还包括图像、图表、视频帧;模型需要在视觉证据上做分解、比较、计算和引用。

与「视觉理解」不同:理解是「看到了什么」,推理是「据此推出什么、证据在哪」。

评测要同时看答案对不对和证据链是否落在图上,否则容易被语言先验蒙混。

长图、多图与视频会显著增加视觉 token 成本,推理步数越多,单价越高。

Inputs include images, charts and video frames, not only text; the model must decompose, compare, calculate and cite visual evidence.

Unlike visual understanding, which asks what is visible, reasoning asks what follows from it and where the evidence sits.

Evaluation must check both the answer and whether the evidence chain lands on the image, or language priors can fake it.

Long images, multiple images and video significantly raise visual-token cost; more reasoning steps mean a higher price.

为什么需要知道Why it matters

2026 年模型都在拼「看图做事」;不会验收多模态推理,就分不清真能力和漂亮话。Models in 2026 compete on seeing and acting; without acceptance tests for multimodal reasoning, real capability is hard to separate from polished talk.

出处Sources
  • OpenAI · Vision guideOpenAI · Vision guide ↗
  • Google · Gemini image understandingGoogle · Gemini image understanding ↗
  • 学 AI · 视觉语言模型Learn AI · Vision-language models ↗