跳转到内容
Alpha
  • 模型
  • 排行榜
  • Ori
联系销售

页脚

OpenRouterOpenRouter
© 2026 OpenRouter, Inc

产品

  • 聊天
  • 排行榜
  • 基准测试
  • 应用
  • 发现
  • 模型
  • Ori
  • 合集
  • 提供商
  • 工具
  • 定价
  • Business
  • Enterprise
  • 实验室

公司

  • 关于
  • 博客
  • 客户
  • 招聘
    招聘
  • 隐私
  • 服务条款
  • 信任中心
  • 支持
  • 兼容 OR
  • 数据
  • 品牌

开发者

  • 文档
  • API 参考
  • 开发者平台
  • 状态
  • AI 站点地图

连接

  • Discord
  • GitHub
  • LinkedIn
  • X
  • YouTube
所有基准测试

VGI-Bench

VGI-Bench(Video General Intelligence Bench,由 Seldon 开发)针对长视频提出单项选择题:什么最先出现、什么与其他事物同时出现在屏幕上、某个特定镜头出现时旁白说了什么,以及前提从未发生过的问题。下方每个样例都是通过 OpenRouter 进行的一次真实运行。模型在采样温度为 0 的情况下,在一轮中接收完整视频和问题,评分所依据的答案是其最后一个 Answer: 行中的字母。

上次基准测试运行 2026年11月2日 23:30 UTC

数据集GitHub
模型对比成本效率排行榜轨迹

模型对比

最准确

bytedance-seed 的网站图标
ByteDance Seed: Seed 1.6

66.2%

最佳性价比

bytedance-seed 的网站图标
ByteDance Seed: Seed 1.6

US$0.039/问题

最快

bytedance-seed 的网站图标
ByteDance Seed: Seed 1.6

64s

准确率
代表性运行的准确率,按从高到低排序。
每个问题的成本
每个问题的平均成本,按从低到高排序。
每题耗时
每道题的平均实际耗时,按耗时从短到长排序。

成本效率

准确率与成本(帕累托前沿)
每个点代表一个模型,在可用时使用默认路由(不固定到某个提供商)。曲线为帕累托前沿:没有模型能在准确率和成本两方面同时优于这些模型。

排行榜

在公开测试集上的准确率(共 439 道题,运行至少需回答 395 道题才会列出),以及模型在各提供商端点上运行时每道题的成本、耗时、输入和输出 token 数。分数汇总了最近 90 天内的成功运行结果。

VGI-Bench 排行榜:按提供商列出模型准确率、成本、耗时及每道题的输入和输出 token 数
#模型标准差
1
bytedance-seed 的网站图标
ByteDance Seed: Seed 1.6
帕累托
66.2%±1.0个百分点US$0.03964s141k2.03k
2
qwen 的网站图标
Qwen: Qwen3.8 Omni Flash
Alibaba
65.5%--US$0.0464.6m412k11.5k
3
bytedance-seed 的网站图标
ByteDance Seed: Seed-2.0-Lite
Seed
63.7%±4.8个百分点US$0.0732.5m241k6.5k
4
qwen 的网站图标
Qwen: Qwen3.8 Max (0902)
Alibaba
63.7%--US$0.558.6m613k20.1k
5
meta 的网站图标
Meta: Muse Wonder
Meta
63.6%--US$1.033.1m233k8.62k
6
google 的网站图标
Google: Gemini 2.5 Pro
Google AI Studio
61.4%±0.7个百分点US$0.0611.8m3416.04k
7
meta 的网站图标
Meta: Muse Spark 1.3
Meta
61.1%--US$0.224.2m227k14.3k
8
qwen 的网站图标
Qwen: Qwen3.8 Max (0803)
Alibaba
58.0%--US$0.338.2m579k13.6k
9
qwen 的网站图标
Qwen: Qwen3.8 Flash
Alibaba
56.8%--US$0.116.2m1.52M24.1k
10
google 的网站图标
Google: Gemini 3 Flash Preview
Google AI Studio
54.8%±0.5个百分点US$0.144.7m111k40.2k
11
qwen 的网站图标
Qwen: Qwen3.7 Flash
Alibaba
54.8%±0.5个百分点US$0.275.6m1.48M10.9k
12
meta 的网站图标
Meta: Muse Spark 1.1
Meta
54.2%±0.5个百分点US$0.242.6m151k21.8k
13
meta 的网站图标
Meta: Muse Spark Open
52.8%±2.3个百分点US$0.163.7m98.6k14.1k
14
google 的网站图标
Google: Gemma 4 31B
帕累托
51.3%--US$0.0086.0m7.53k13.7k
15
meta 的网站图标
Meta: Muse Spark 1.2
Meta
51.0%±0.6个百分点US$0.162.0m150k15.3k
16
qwen 的网站图标
Qwen: Qwen3.6 Plus
Alibaba
50.6%±0.4个百分点US$0.217.1m532k20.9k
17
google 的网站图标
Google: Gemini 2.5 Flash
Google AI Studio
50.0%±0.3个百分点US$0.0241.9m3419.42k
18
google 的网站图标
Google: Gemini 2.5 Flash Lite
Google AI Studio
48.0%--US$0.0122.4m34029.7k
19
qwen 的网站图标
Qwen: Qwen3.8 27B
47.6%±1.9个百分点US$0.0262.3m36.7k7.15k
20
xiaomi 的网站图标
Xiaomi: MiMo-V2.5
Xiaomi
46.5%±2.9个百分点US$0.0186.4m272k13.2k
21
perceptron 的网站图标
Perceptron: Perceptron Mk1
Perceptron
46.2%±0.1个百分点US$0.0082.7m46.4k817
22
qwen 的网站图标
Qwen: Qwen3.5-Flash
Alibaba
46.2%±1.2个百分点US$0.104.9m1.49M20.2k
23
xiaomi 的网站图标
Xiaomi: MiMo-X-Pro Preview
Xiaomi
45.6%--US$0.0274.6m77.4k14.7k
24
qwen 的网站图标
Qwen: Qwen3.5 397B A17B
45.5%±1.0个百分点US$0.958.6m1.45M21.4k
25
qwen 的网站图标
Qwen: Qwen3.5 Plus 2026-02-15
Alibaba
45.5%±0.7个百分点US$0.177.5m532k21.3k
26
xiaomi 的网站图标
Xiaomi: MiMo-X-Flash Preview
Xiaomi
45.4%--US$0.0345.2m337k11.9k
27
qwen 的网站图标
Qwen: Qwen3.6 Flash
Alibaba
45.2%±0.8个百分点US$1.146.7m1.49M19.4k
28
qwen 的网站图标
Qwen: Qwen3.5 Plus 2026-04-20
Alibaba
45.1%±0.6个百分点US$0.174.9m440k18.6k
29
xiaomi 的网站图标
Xiaomi: MiMo-V2.6-Pro
44.9%±0.8个百分点US$0.0356.9m77.3k18.5k
30
bytedance-seed 的网站图标
ByteDance Seed: Seed-2.0-Mini
Seed
43.4%±2.9个百分点US$0.0271.9m232k9.34k
31
xiaomi 的网站图标
Xiaomi: MiMo-V2.6-Pro-UltraSpeed
Xiaomi
43.4%--US$0.343.2m77k21.1k
32
xiaomi 的网站图标
Xiaomi: MiMo-V2.6-Flash
35.5%±3.4个百分点US$0.0165.2m150k12k
33
z-ai 的网站图标
Z.ai: GLM 5.3 Flash
34.6%±2.7个百分点US$0.0288.3m58.2k43.9k
34
minimax 的网站图标
MiniMax: MiniMax M3
30.1%±2.6个百分点US$0.0819.8m53.4k75.3k
35
perceptron 的网站图标
Perceptron: Perceptron Mk1.5
Perceptron
28.9%--US$0.0191.6m49k7.48k
36
nvidia 的网站图标
NVIDIA: Nemotron 3.5 Super VL
28.5%±0.8个百分点US$0.0161.9m14.6k30.3k
37
google 的网站图标
Google: Gemma 4 26B A4B
帕累托
24.7%±7.3个百分点US$0.0042.8m4.59k9.91k
38
minimax 的网站图标
MiniMax: MiniMax M3.1 Flash
22.2%--US$0.00644s130k3.99k
39
rekaai 的网站图标
Reka Edge
帕累托
Reka
21.4%±0.2个百分点US$0.0001531s1.49k30.5
40
inclusionai 的网站图标
inclusionAI: Ling 3.0 Flash VL
19.2%--US$0.0011.8m5.71k9.46k
41
google 的网站图标
Google: Gemini 4 Argon
15.5%--US$0.05044s1.58k4.7k
42
google 的网站图标
Google: Gemini 4 Carbon (EAP)
Google AI Studio
14.7%--US$0.323.0m1.63k15.5k
43
google 的网站图标
Google: Gemini 3.8 Flash
14.4%--US$0.03976s1.42k10.3k
44
elorian 的网站图标elorian 的网站图标
Elorian: Glimpse
Together
13.8%--US$0.02247s6.28k6.46k
45
qwen 的网站图标
Qwen: Qwen3.5-122B-A10B
Alibaba
12.8%±0.4个百分点US$0.0233.9m38k6.22k
46
google 的网站图标
Google: Gemini 3.7 Flash
Google
12.4%--US$0.01324s1.19k3.29k
47
qwen 的网站图标
Qwen: Qwen3.5-35B-A3B
12.4%±0.4个百分点US$0.01556s38.3k6.56k
48
z-ai 的网站图标
Z.ai: GLM 5V Turbo
Z.AI
12.3%±0.5个百分点US$0.136.9m124k2.42k
49
qwen 的网站图标
Qwen: Qwen3.6 27B
11.8%±0.4个百分点US$0.0313.9m32.7k5.87k
50
google 的网站图标
Google: Gemini 3.6 Flash
Google AI Studio
11.5%--US$0.01624s1.39k3.97k
51
google 的网站图标
Google: Gemini 3.1 Pro Preview
Google AI Studio
10.9%--US$0.09859s1.39k7.96k
52
qwen 的网站图标
Qwen: Qwen3.6 35B A3B
10.8%±0.3个百分点US$0.00544s4.57k5.08k
53
google 的网站图标
Google: Gemini 3.1 Pro Preview Custom Tools
Google AI Studio
10.6%--US$0.09860s1.39k8.02k
54
google 的网站图标
Google: Gemini 3.5 Flash
Google AI Studio
10.1%--US$0.08839s1.38k9.59k
55
google 的网站图标
Google: Gemini 3.5 Flash Lite
Google AI Studio
8.4%--US$0.00613s1.39k2.08k
56
google 的网站图标
Google: Gemini 3.1 Flash Lite
Google AI Studio
8.2%--US$0.03858s1.39k25.1k
57
google 的网站图标
Google: Gemini 3.1 Flash Lite Preview
Google AI Studio
8.2%--US$0.03858s1.38k25.1k
58
moonshotai 的网站图标
MoonshotAI: Kimi K3
6.3%±5.7个百分点US$0.02536s5.46k1.69k
59
z-ai 的网站图标
Z.ai: GLM 4.6V
Z.AI
5.6%±4.3个百分点US$0.0171.9m69.2k804
60
qwen 的网站图标
Qwen: Qwen3.5-9B
2.7%±1.2个百分点US$0.0038.8m61223.6k

轨迹

推理轨迹体现了模型的感知:它声称看到的片段,以及为这些片段标注的时间戳。将推理轨迹与视频对照阅读,可以了解答案是如何得出的,以及在失败的运行中,描述在何处偏离了视频画面。

正确

指针悬停时间最长的窗口

正常流程: 模型定位到了该时刻,并正确解读了它。

问题 1187 global-graph/interaction_partner Qwen3.8 27B

发送给模型的完整视频,由智能体运行框架所使用的 OpenRouter 镜像提供。

鼠标指针最常悬停在哪个选项上?

  1. A)Windhawk application window密钥,模型
  2. B)Install button
  3. C)Adobe Premiere Pro window
  4. D)Windows taskbar
回答了 A ✓按键 A

该追踪会在整个录制过程中跟踪指针,而不是只看一帧,将接近结尾时短暂出现的 Premiere Pro 标为干扰项,并最终确定在整个会话中占据主导地位的窗口。

The question asks which option the mouse pointer hovers over most often. Let me think about the video content. The video is a tutorial about Windhawk, a customization tool for Windows. Throughout the video, the user is constantly interacting with the Windhawk application window - browsing mods, installing them, configuring settings, etc. Let me consider each option: - A) Windhawk application window - This is the main focus of the video. The user opens Windhawk, browses mods, clicks on details, settings, installs mods, etc. The mouse is frequently over this window. - B) Install button - The install button is clicked a few times but not hovered over most often. - C) Adobe Premiere Pro window - This appears only briefly near the end (around 13:20-13:40) as a joke about the title bar color. - D) Windows taskbar - The taskbar is shown but the mouse isn't constantly hovering over it. The dominant interaction is with the Windhawk application window. The mouse pointer is over it for the majority of the video as the user navigates through mods, settings, etc. So the answer is A. Let me double check by reviewing timestamps. From 1:20 onwards, the Windhawk window is open and the mouse moves around it constantly - clicking Mods, Explore, scrolling, clicking Details, Settings, etc. This continues through most of the video. The taskbar is shown at times but the mouse hovers over the Windhawk window more. Thus A is correct.

错误

与花园棚屋同时出现的物体

失败模式: 按键上方的显著物体

问题 1862 global-graph/co_presence Qwen3.8 27B

发送给模型的完整视频,由智能体运行框架所使用的 OpenRouter 镜像提供。

视频中,哪个选项反复与木制花园棚屋同时出现?

  1. A)presenter with short curly hair键
  2. B)metal shed hinge
  3. C)wagon wheel模型
  4. D)glass doors
回答了 C ✗按键 A

该轨迹引用了八个时间戳,指向靠在棚屋旁的马车车轮,并将出镜讲解者视为无关紧要。人工验证的标准答案是出镜讲解者。模型选择了画面中最显眼的物体,而非题目评分所依据的对象。

The question asks which option is repeatedly visible at the same time as the wooden garden shed during the video. Let me look at the frames where a wooden garden shed appears. At around 3:57-4:00, there's a wooden garden shed with a wagon wheel leaning against it, and the presenter is sitting in front of it. At 4:57-5:15, the same wooden shed with the wagon wheel is visible while the presenter talks about the slab base. So the wooden garden shed appears together with the wagon wheel in multiple segments. Let me check the other options: - A) presenter with short curly hair - the presenter is visible in many places, but the question is about what's visible at the same time as the wooden shed. The presenter is there too, but the wagon wheel is a distinctive object attached to the shed scene. - B) metal shed hinge - not particularly visible. - C) wagon wheel - yes, clearly visible leaning against the wooden shed in multiple frames (3:57, 4:00, 4:57, 5:00, 5:03, 5:07, 5:11, 5:15). - D) glass doors - those are on a different building (the dark shed with white doors). The wagon wheel is repeatedly visible alongside the wooden garden shed. The answer is C. Let me double check the frames. At 3:57 the presenter sits against a wooden shed with a large wagon wheel on the right. At 4:57-5:15 same scene. So wagon wheel is repeatedly visible with the wooden shed. Thus answer C.

错误

一个关于量匙的错误前提问题

失败模式: 叙述从未发生的瞬间

问题 1096 contrastive-hard-negative/false-premise-visual-attribute Qwen3.8 27B

发送给模型的完整视频,由智能体运行框架所使用的 OpenRouter 镜像提供。

当说话者说她用了一满汤匙盐时,红色量勺位于奶酪凝块的什么位置?

  1. A)This does not happen in the video.键
  2. B)It is resting beside the bowl of cheese curds.
  3. C)It is positioned just in front of the cheese curds.模型
  4. D)It is laid across the top of the cheese curds.
回答了 C ✗按键 A

标准答案指出,这一时刻并未出现。轨迹描述了从 2:38 到 2:44 的一段过程,其中勺子被放在凝乳前方,直接否定了“未发生”选项,并给出了标准答案不支持的细节。

The user wants me to look at the video and determine the position of the red measuring spoon relative to the cheese curds when the speaker mentions using a full tablespoon of salt. Let me trace through the video. Around 2:38-2:42, the speaker is adding salt to the cheese curds in a cloth-lined pot. She holds a red measuring spoon with salt in it. At 2:42, she sets the red spoon down. Let me look at where it ends up. At 2:42-2:44, after adding the salt, she places the red spoon down in front of the bowl/curds. Looking at the frame at 2:44, the red spoon is lying on the counter just in front of the cheese curds (the cloth-lined pot). Then she picks up the wooden spoon to stir. Let me check the options: A) Does not happen - but it does happen, she uses a tablespoon of salt. B) Resting beside the bowl - "beside" could be left/right. C) Positioned just in front of the cheese curds. D) Laid across the top. Looking at the frame at 2:44, the red spoon is at the bottom of the frame, in front of the curds. So "just in front of the cheese curds" matches C. Let me double check the timestamp. At 2:38 she holds the red spoon with salt. At 2:42 she tips it in. At 2:44 the red spoon is resting on the counter at the front edge, in front of the curds. Yes. Thus answer C. I need to provide reasoning then final answer format.

错误

M.2 散热片锁定的声音

失败模式: 根据世界知识回答音频问题

问题 2852 contrastive-hard-negative/visual_audio Qwen3.8 27B

发送给模型的完整视频,由智能体运行框架所使用的 OpenRouter 镜像提供。

当两个 M.2 插槽以紫色高亮显示时,顶部散热片锁回原位时会发出什么咔嗒声?

  1. A)This does not happen in the video键
  2. B)A sharp metallic snap
  3. C)A firm double-click
  4. D)A soft plastic click模型
回答了 D ✗按键 A

追踪中写道“我需要回忆音频”,随后根据电脑装机视频“通常”听起来是什么样来推理,而完全没有依据这段视频。关键在于“没有发生”。最终回复将猜测的咔嗒声描述为在视频中听到的声音。

The user is asking about a specific audio detail in the video. Let me look at the relevant section. Around 02:20-02:30, the video shows the M.2 slots being highlighted in purple (the diagram at 02:20 shows the M.2 slots). Then the person is working with the heatsink. Let me think about the sequence. At 02:20 there's a diagram showing M.2 slots highlighted. Then at 02:30 the person is holding a black heatsink piece. At 02:40-02:50 they're working on the motherboard with the M.2 area. The question asks: when the two M.2 slots are highlighted in purple, what click is heard as the top heatsink locks back into place? I need to recall the audio. In these PC build videos, when you press down on the M.2 heatsink, there's typically a plastic click as it locks. The options are: sharp metallic snap, firm double-click, soft plastic click, or doesn't happen. Given the context of an M.2 heatsink locking into place, the typical sound is a soft plastic click. The heatsink is plastic/metal but the locking mechanism is usually a plastic clip. Let me consider the diagram at 02:20 - it shows the M.2 slots with purple/magenta highlighting. Then the person installs the SSD and the heatsink. The click when the heatsink locks would be a soft plastic click. I'll go with D, soft plastic click. Let me double check by reasoning about the video flow. The diagram with purple highlights appears at 02:20. Then at 02:30-02:50 the person is handling the heatsink and the M.2 area. The locking click of an M.2 heatsink is generally a soft plastic click. Thus answer D.

错误

正在插入的电源线后方的背景

失败模式: 定位近乎重复的片段

问题 1539 near-duplicate-moments Qwen3.8 27B

发送给模型的完整视频,由智能体运行框架所使用的 OpenRouter 镜像提供。

当手将黑色电源线插入黑色 PC 机箱后部的 Seasonic 电源时,其后方是什么表面?

  1. A)A dark, textured surface键
  2. B)A gray carpeted surface
  3. C)A metal mesh surface
  4. D)A plain white surface模型
回答了 D ✗按键 A

问题锁定的是一个反复出现但略有差异的瞬间。轨迹最终选定了 15:30 至 15:40 的背面视角,其后方是一面白墙。关键是一处深色纹理表面,因此模型将该动作匹配到了一个相似镜头,并从那个镜头中读取了背景。

The user wants me to identify the surface behind the action of plugging a black power cable into the Seasonic PSU at the rear of the case. Let me look at the relevant timestamps. Around 15:30-15:40, the video shows the rear of the case with the Seasonic PSU installed, and a hand plugging in a black power cable (the IEC power cord) into the PSU. Looking at the frames around 15:30-15:40, the background behind the case appears to be a plain white wall/surface. There's a white wall with some light switches visible in earlier frames (around 7:50-8:00), and at the rear view (15:30-15:40) the background is a plain white surface. Let me verify by examining the frames. At 15:30, the case is shown from the rear with the Seasonic logo, and the background is white. At 15:40, the hand plugs the cable in, and the background is a plain white surface. So the answer is D) A plain white surface. Let me double check the other options: A) dark textured surface - that's the table surface where components were laid out earlier. B) gray carpeted surface - similar to the table. C) metal mesh surface - that's part of the case itself. The background behind the case at the rear view is white. Thus D.

错误

说出某句台词时威士忌酒瓶的位置

失败模式: 始终未能定位到口头提示

问题 2894 contrastive-hard-negative/transcript_visual Qwen3.8 27B

发送给模型的完整视频,由智能体运行框架所使用的 OpenRouter 镜像提供。

当他说“这段时间他一直在后面喝威士忌”时,威士忌酒瓶位于 Craig 工作台上的笔记本电脑左侧还是右侧?

  1. A)This does not happen in the video.键
  2. B)To the left of the laptop.模型
  3. C)To the right of the laptop.
  4. D)Directly behind the laptop.
回答了 B ✗按键 A

线索是一句台词。追踪始终未找到这句台词,称其“可能出现在长椅旁的对话中”,并根据 60 到 70 秒处的画面作答。关键在于“并未发生”。缺少转录文本中的定位依据,模型便用最接近的场景代替。

The user is asking about a specific moment in the video where someone says "This whole time he's back there drinking whiskey." I need to find this moment and look at the position of the whiskey bottle relative to the laptop on Craig's bench. Let me look at the early part of the video. Around 60-70 seconds, there's a scene with a man sitting at a bench with a laptop, and there's a bottle on the bench. Let me look carefully. At around 60-70 seconds, there's a man in a light blue shirt sitting in a chair at a bench with a laptop. On the bench, to the left of the laptop (from the viewer's perspective), there's a bottle (looks like a whiskey bottle) and a mug. Wait, I need to find the audio line "This whole time he's back there drinking whiskey." This is likely during the scene where two people are at the bench with the laptop. Let me look at the timestamps around 60-130 seconds. At 60-70s: man in blue shirt at bench with laptop, bottle on left side of bench. At 120-130s: close up of laptop showing a 3D model of a table, with a hand typing. The line about drinking whiskey probably occurs during the conversation at the bench. The bottle is on the bench. Looking at the frame at 60-70s, the bottle is to the left of the laptop (viewer's left). The laptop is on the right side of the bench, the bottle and mug are on the left side. So from the viewer's perspective, the whiskey bottle is to the left of the laptop. Thus the answer is B. Let me double check the orientation. The question asks "to the left or right of the laptop on Craig's bench?" In the image at 60-70s, the laptop is on the right portion of the bench, the bottle is on the left portion. So the bottle is to the left of the laptop. Therefore answer B. I'm fairly confident. The line "This whole time he's back there drinking whiskey" matches the scene where one person is at the laptop and another is "back there" drinking. The bottle visible on the bench is to the left of the laptop. Thus B.