Pular para o conteúdo
Alfa
  • Modelos
  • Classificações
  • Ori
Falar com vendas

Rodapé

OpenRouterOpenRouter
© 2026 OpenRouter, Inc

Produto

  • Chat
  • Classificações
  • Avaliações de desempenho
  • Aplicativos
  • Descobrir
  • Modelos
  • Ori
  • Coleções
  • Provedores
  • Ferramentas
  • Preços
  • Business
  • Enterprise
  • Laboratório

Empresa

  • Sobre
  • Blog
  • Clientes
  • Carreiras
    Vagas
  • Privacidade
  • Termos de serviço
  • Central de confiança
  • Suporte
  • Funciona com OR
  • Dados
  • Marca

Desenvolvedor

  • Documentação
  • Referência da API
  • Plataforma para desenvolvedores
  • Status
  • Mapa do site para IA

Conectar

  • Discord
  • GitHub
  • LinkedIn
  • X
  • YouTube
Todos os testes de desempenho

VGI-Bench

O VGI-Bench (Video General Intelligence Bench, da Seldon) faz perguntas de múltipla escolha sobre vídeos longos: o que apareceu primeiro, o que estava na tela ao mesmo tempo que outra coisa, o que um narrador disse enquanto uma cena específica era exibida e perguntas cuja premissa nunca acontece. Cada exemplo abaixo é uma execução real pela OpenRouter. O modelo recebeu o vídeo completo e a pergunta em um único turno com temperatura 0, e a resposta avaliada é a letra em sua última linha Answer: .

Última execução da avaliação comparativa 2 de nov. de 2026, 23:30 UTC

Conjunto de dadosGitHub
Comparação de modelosEficiência de custoClassificaçãoTrajetórias

Comparação de modelos

Mais preciso

Favicon de bytedance-seed
ByteDance Seed: Seed 1.6

66,2%

Melhor custo-benefício

Favicon de bytedance-seed
ByteDance Seed: Seed 1.6

US$ 0,039/pergunta

Mais rápido

Favicon de bytedance-seed
ByteDance Seed: Seed 1.6

64s

Acurácia
Acurácia da execução representativa, da melhor para a pior.
Custo por pergunta
Custo médio por pergunta, do menor para o maior.
Tempo por pergunta
Tempo médio decorrido por pergunta, do mais rápido para o mais lento.

Eficiência de custos

Precisão vs. custo (fronteira de Pareto)
Um ponto por modelo, usando o roteamento padrão (sem fixar um provedor) quando disponível. A linha é a fronteira de Pareto: nenhum modelo supera esses modelos tanto em precisão quanto em custo.

Classificação

Acurácia no subconjunto público (439 perguntas, as execuções precisam ter pelo menos 395 respondidas para serem listadas), com custo, tempo e tokens de entrada e saída por pergunta para cada endpoint de provedor no qual o modelo foi executado. As pontuações agregam execuções bem-sucedidas dos últimos 90 dias.

Classificação do VGI-Bench: precisão do modelo, custo, tempo e tokens de entrada e saída por pergunta, por provedor
#ModeloDesvio padrão
1
Favicon de bytedance-seed
ByteDance Seed: Seed 1.6
Pareto
66,2%±1,0ppUS$ 0,03964s141k2,03k
2
Favicon de qwen
Qwen: Qwen3.8 Omni Flash
Alibaba
65,5%--US$ 0,0464,6m412k11,5k
3
Favicon de bytedance-seed
ByteDance Seed: Seed-2.0-Lite
Seed
63,7%±4,8ppUS$ 0,0732,5m241k6,5k
4
Favicon de qwen
Qwen: Qwen3.8 Max (0902)
Alibaba
63,7%--US$ 0,558,6m613k20,1k
5
Favicon de meta
Meta: Muse Wonder
Meta
63,6%--US$ 1,033,1m233k8,62k
6
Favicon de google
Google: Gemini 2.5 Pro
Google AI Studio
61,4%±0,7ppUS$ 0,0611,8m3416,04k
7
Favicon de meta
Meta: Muse Spark 1.3
Meta
61,1%--US$ 0,224,2m227k14,3k
8
Favicon de qwen
Qwen: Qwen3.8 Max (0803)
Alibaba
58,0%--US$ 0,338,2m579k13,6k
9
Favicon de qwen
Qwen: Qwen3.8 Flash
Alibaba
56,8%--US$ 0,116,2m1,52M24,1k
10
Favicon de google
Google: Gemini 3 Flash Preview
Google AI Studio
54,8%±0,5ppUS$ 0,144,7m111k40,2k
11
Favicon de qwen
Qwen: Qwen3.7 Flash
Alibaba
54,8%±0,5ppUS$ 0,275,6m1,48M10,9k
12
Favicon de meta
Meta: Muse Spark 1.1
Meta
54,2%±0,5ppUS$ 0,242,6m151k21,8k
13
Favicon de meta
Meta: Muse Spark Open
52,8%±2,3ppUS$ 0,163,7m98,6k14,1k
14
Favicon de google
Google: Gemma 4 31B
Pareto
51,3%--US$ 0,0086,0m7,53k13,7k
15
Favicon de meta
Meta: Muse Spark 1.2
Meta
51,0%±0,6ppUS$ 0,162,0m150k15,3k
16
Favicon de qwen
Qwen: Qwen3.6 Plus
Alibaba
50,6%±0,4ppUS$ 0,217,1m532k20,9k
17
Favicon de google
Google: Gemini 2.5 Flash
Google AI Studio
50,0%±0,3ppUS$ 0,0241,9m3419,42k
18
Favicon de google
Google: Gemini 2.5 Flash Lite
Google AI Studio
48,0%--US$ 0,0122,4m34029,7k
19
Favicon de qwen
Qwen: Qwen3.8 27B
47,6%±1,9ppUS$ 0,0262,3m36,7k7,15k
20
Favicon de xiaomi
Xiaomi: MiMo-V2.5
Xiaomi
46,5%±2,9ppUS$ 0,0186,4m272k13,2k
21
Favicon de perceptron
Perceptron: Perceptron Mk1
Perceptron
46,2%±0,1ppUS$ 0,0082,7m46,4k817
22
Favicon de qwen
Qwen: Qwen3.5-Flash
Alibaba
46,2%±1,2ppUS$ 0,104,9m1,49M20,2k
23
Favicon de xiaomi
Xiaomi: MiMo-X-Pro Preview
Xiaomi
45,6%--US$ 0,0274,6m77,4k14,7k
24
Favicon de qwen
Qwen: Qwen3.5 397B A17B
45,5%±1,0ppUS$ 0,958,6m1,45M21,4k
25
Favicon de qwen
Qwen: Qwen3.5 Plus 2026-02-15
Alibaba
45,5%±0,7ppUS$ 0,177,5m532k21,3k
26
Favicon de xiaomi
Xiaomi: MiMo-X-Flash Preview
Xiaomi
45,4%--US$ 0,0345,2m337k11,9k
27
Favicon de qwen
Qwen: Qwen3.6 Flash
Alibaba
45,2%±0,8ppUS$ 1,146,7m1,49M19,4k
28
Favicon de qwen
Qwen: Qwen3.5 Plus 2026-04-20
Alibaba
45,1%±0,6ppUS$ 0,174,9m440k18,6k
29
Favicon de xiaomi
Xiaomi: MiMo-V2.6-Pro
44,9%±0,8ppUS$ 0,0356,9m77,3k18,5k
30
Favicon de bytedance-seed
ByteDance Seed: Seed-2.0-Mini
Seed
43,4%±2,9ppUS$ 0,0271,9m232k9,34k
31
Favicon de xiaomi
Xiaomi: MiMo-V2.6-Pro-UltraSpeed
Xiaomi
43,4%--US$ 0,343,2m77k21,1k
32
Favicon de xiaomi
Xiaomi: MiMo-V2.6-Flash
35,5%±3,4ppUS$ 0,0165,2m150k12k
33
Favicon de z-ai
Z.ai: GLM 5.3 Flash
34,6%±2,7ppUS$ 0,0288,3m58,2k43,9k
34
Favicon de minimax
MiniMax: MiniMax M3
30,1%±2,6ppUS$ 0,0819,8m53,4k75,3k
35
Favicon de perceptron
Perceptron: Perceptron Mk1.5
Perceptron
28,9%--US$ 0,0191,6m49k7,48k
36
Favicon de nvidia
NVIDIA: Nemotron 3.5 Super VL
28,5%±0,8ppUS$ 0,0161,9m14,6k30,3k
37
Favicon de google
Google: Gemma 4 26B A4B
Pareto
24,7%±7,3ppUS$ 0,0042,8m4,59k9,91k
38
Favicon de minimax
MiniMax: MiniMax M3.1 Flash
22,2%--US$ 0,00644s130k3,99k
39
Favicon de rekaai
Reka Edge
Pareto
Reka
21,4%±0,2ppUS$ 0,0001531s1,49k30,5
40
Favicon de inclusionai
inclusionAI: Ling 3.0 Flash VL
19,2%--US$ 0,0011,8m5,71k9,46k
41
Favicon de google
Google: Gemini 4 Argon
15,5%--US$ 0,05044s1,58k4,7k
42
Favicon de google
Google: Gemini 4 Carbon (EAP)
Google AI Studio
14,7%--US$ 0,323,0m1,63k15,5k
43
Favicon de google
Google: Gemini 3.8 Flash
14,4%--US$ 0,03976s1,42k10,3k
44
Favicon de elorianFavicon de elorian
Elorian: Glimpse
Together
13,8%--US$ 0,02247s6,28k6,46k
45
Favicon de qwen
Qwen: Qwen3.5-122B-A10B
Alibaba
12,8%±0,4ppUS$ 0,0233,9m38k6,22k
46
Favicon de google
Google: Gemini 3.7 Flash
Google
12,4%--US$ 0,01324s1,19k3,29k
47
Favicon de qwen
Qwen: Qwen3.5-35B-A3B
12,4%±0,4ppUS$ 0,01556s38,3k6,56k
48
Favicon de z-ai
Z.ai: GLM 5V Turbo
Z.AI
12,3%±0,5ppUS$ 0,136,9m124k2,42k
49
Favicon de qwen
Qwen: Qwen3.6 27B
11,8%±0,4ppUS$ 0,0313,9m32,7k5,87k
50
Favicon de google
Google: Gemini 3.6 Flash
Google AI Studio
11,5%--US$ 0,01624s1,39k3,97k
51
Favicon de google
Google: Gemini 3.1 Pro Preview
Google AI Studio
10,9%--US$ 0,09859s1,39k7,96k
52
Favicon de qwen
Qwen: Qwen3.6 35B A3B
10,8%±0,3ppUS$ 0,00544s4,57k5,08k
53
Favicon de google
Google: Gemini 3.1 Pro Preview Custom Tools
Google AI Studio
10,6%--US$ 0,09860s1,39k8,02k
54
Favicon de google
Google: Gemini 3.5 Flash
Google AI Studio
10,1%--US$ 0,08839s1,38k9,59k
55
Favicon de google
Google: Gemini 3.5 Flash Lite
Google AI Studio
8,4%--US$ 0,00613s1,39k2,08k
56
Favicon de google
Google: Gemini 3.1 Flash Lite
Google AI Studio
8,2%--US$ 0,03858s1,39k25,1k
57
Favicon de google
Google: Gemini 3.1 Flash Lite Preview
Google AI Studio
8,2%--US$ 0,03858s1,38k25,1k
58
Favicon de moonshotai
MoonshotAI: Kimi K3
6,3%±5,7ppUS$ 0,02536s5,46k1,69k
59
Favicon de z-ai
Z.ai: GLM 4.6V
Z.AI
5,6%±4,3ppUS$ 0,0171,9m69,2k804
60
Favicon de qwen
Qwen: Qwen3.5-9B
2,7%±1,2ppUS$ 0,0038,8m61223,6k

Trajetórias

O rastro de raciocínio é a percepção do modelo: os momentos que ele diz ter visto e os registros de tempo que associa a eles. Compará-lo com o vídeo mostra como a resposta foi alcançada e, nas execuções que falharam, onde a descrição diverge das imagens.

correto

Sobre qual janela o ponteiro permanece por mais tempo

Caminho ideal: O modelo localiza o momento e o interpreta corretamente.

Pergunta 1187 global-graph/interaction_partner Qwen3.8 27B

O vídeo completo enviado ao modelo, disponibilizado pelo espelho do OpenRouter usado pelo harness.

Sobre qual opção o ponteiro do mouse fica com mais frequência?

  1. A)Windhawk application windowchave, modelo
  2. B)Install button
  3. C)Adobe Premiere Pro window
  4. D)Windows taskbar
respondeu A ✓chave A

O rastro acompanha o ponteiro ao longo de toda a gravação, em vez de apenas um quadro, identifica a breve aparição do Premiere Pro perto do final como uma distração e chega à janela que domina a sessão.

The question asks which option the mouse pointer hovers over most often. Let me think about the video content. The video is a tutorial about Windhawk, a customization tool for Windows. Throughout the video, the user is constantly interacting with the Windhawk application window - browsing mods, installing them, configuring settings, etc. Let me consider each option: - A) Windhawk application window - This is the main focus of the video. The user opens Windhawk, browses mods, clicks on details, settings, installs mods, etc. The mouse is frequently over this window. - B) Install button - The install button is clicked a few times but not hovered over most often. - C) Adobe Premiere Pro window - This appears only briefly near the end (around 13:20-13:40) as a joke about the title bar color. - D) Windows taskbar - The taskbar is shown but the mouse isn't constantly hovering over it. The dominant interaction is with the Windhawk application window. The mouse pointer is over it for the majority of the video as the user navigates through mods, settings, etc. So the answer is A. Let me double check by reviewing timestamps. From 1:20 onwards, the Windhawk window is open and the mouse moves around it constantly - clicking Mods, Explore, scrolling, clicking Details, Settings, etc. This continues through most of the video. The taskbar is shown at times but the mouse hovers over the Windhawk window more. Thus A is correct.

incorreto

Objeto presente junto a um abrigo de jardim

Modo de falha: Objeto saliente sobre a chave

Pergunta 1862 global-graph/co_presence Qwen3.8 27B

O vídeo completo enviado ao modelo, disponibilizado pelo espelho do OpenRouter usado pelo harness.

Qual opção aparece repetidamente ao mesmo tempo que o galpão de madeira do jardim durante o vídeo?

  1. A)presenter with short curly hairchave
  2. B)metal shed hinge
  3. C)wagon wheelmodelo
  4. D)glass doors
respondeu C ✗chave A

O rastro cita oito marcações de tempo de uma roda de carroça encostada no galpão e considera o apresentador incidental. A resposta correta validada por humanos é o apresentador. O modelo escolheu o objeto mais distinto no quadro em vez daquele usado para pontuar a questão.

The question asks which option is repeatedly visible at the same time as the wooden garden shed during the video. Let me look at the frames where a wooden garden shed appears. At around 3:57-4:00, there's a wooden garden shed with a wagon wheel leaning against it, and the presenter is sitting in front of it. At 4:57-5:15, the same wooden shed with the wagon wheel is visible while the presenter talks about the slab base. So the wooden garden shed appears together with the wagon wheel in multiple segments. Let me check the other options: - A) presenter with short curly hair - the presenter is visible in many places, but the question is about what's visible at the same time as the wooden shed. The presenter is there too, but the wagon wheel is a distinctive object attached to the shed scene. - B) metal shed hinge - not particularly visible. - C) wagon wheel - yes, clearly visible leaning against the wooden shed in multiple frames (3:57, 4:00, 4:57, 5:00, 5:03, 5:07, 5:11, 5:15). - D) glass doors - those are on a different building (the dark shed with white doors). The wagon wheel is repeatedly visible alongside the wooden garden shed. The answer is C. Let me double check the frames. At 3:57 the presenter sits against a wooden shed with a large wagon wheel on the right. At 4:57-5:15 same scene. So wagon wheel is repeatedly visible with the wooden shed. Thus answer C.

incorreto

Uma pergunta com premissa falsa sobre uma colher medidora

Modo de falha: Narra um momento que nunca acontece

Pergunta 1096 contrastive-hard-negative/false-premise-visual-attribute Qwen3.8 27B

O vídeo completo enviado ao modelo, disponibilizado pelo espelho do OpenRouter usado pelo harness.

Quando a falante diz que usou uma colher de sopa cheia de sal, onde está posicionada a colher medidora vermelha em relação aos grãos de coalhada?

  1. A)This does not happen in the video.chave
  2. B)It is resting beside the bowl of cheese curds.
  3. C)It is positioned just in front of the cheese curds.modelo
  4. D)It is laid across the top of the cheese curds.
respondeu C ✗chave A

O gabarito diz que esse momento não ocorre. O rastreamento narra uma sequência de 2:38 a 2:44 com a colher colocada em frente à coalhada, rejeita categoricamente a opção 'não acontece' e responde com detalhes que o gabarito não sustenta.

The user wants me to look at the video and determine the position of the red measuring spoon relative to the cheese curds when the speaker mentions using a full tablespoon of salt. Let me trace through the video. Around 2:38-2:42, the speaker is adding salt to the cheese curds in a cloth-lined pot. She holds a red measuring spoon with salt in it. At 2:42, she sets the red spoon down. Let me look at where it ends up. At 2:42-2:44, after adding the salt, she places the red spoon down in front of the bowl/curds. Looking at the frame at 2:44, the red spoon is lying on the counter just in front of the cheese curds (the cloth-lined pot). Then she picks up the wooden spoon to stir. Let me check the options: A) Does not happen - but it does happen, she uses a tablespoon of salt. B) Resting beside the bowl - "beside" could be left/right. C) Positioned just in front of the cheese curds. D) Laid across the top. Looking at the frame at 2:44, the red spoon is at the bottom of the frame, in front of the curds. So "just in front of the cheese curds" matches C. Let me double check the timestamp. At 2:38 she holds the red spoon with salt. At 2:42 she tips it in. At 2:44 the red spoon is resting on the counter at the front edge, in front of the curds. Yes. Thus answer C. I need to provide reasoning then final answer format.

incorreto

O som de um dissipador de calor M.2 se encaixando

Modo de falha: Responde a uma pergunta em áudio com base em conhecimentos sobre o mundo

Pergunta 2852 contrastive-hard-negative/visual_audio Qwen3.8 27B

O vídeo completo enviado ao modelo, disponibilizado pelo espelho do OpenRouter usado pelo harness.

Quando os dois slots M.2 estão destacados em roxo, que clique é ouvido quando o dissipador de calor superior se encaixa novamente no lugar?

  1. A)This does not happen in the videochave
  2. B)A sharp metallic snap
  3. C)A firm double-click
  4. D)A soft plastic clickmodelo
respondeu D ✗chave A

O rastreamento diz 'preciso me lembrar do áudio' e depois raciocina com base em como vídeos de montagem de PCs 'tipicamente' soam, nunca com base neste vídeo. O ponto-chave é 'não acontece'. A resposta final apresenta o clique presumido como algo ouvido no vídeo.

The user is asking about a specific audio detail in the video. Let me look at the relevant section. Around 02:20-02:30, the video shows the M.2 slots being highlighted in purple (the diagram at 02:20 shows the M.2 slots). Then the person is working with the heatsink. Let me think about the sequence. At 02:20 there's a diagram showing M.2 slots highlighted. Then at 02:30 the person is holding a black heatsink piece. At 02:40-02:50 they're working on the motherboard with the M.2 area. The question asks: when the two M.2 slots are highlighted in purple, what click is heard as the top heatsink locks back into place? I need to recall the audio. In these PC build videos, when you press down on the M.2 heatsink, there's typically a plastic click as it locks. The options are: sharp metallic snap, firm double-click, soft plastic click, or doesn't happen. Given the context of an M.2 heatsink locking into place, the typical sound is a soft plastic click. The heatsink is plastic/metal but the locking mechanism is usually a plastic clip. Let me consider the diagram at 02:20 - it shows the M.2 slots with purple/magenta highlighting. Then the person installs the SSD and the heatsink. The click when the heatsink locks would be a soft plastic click. I'll go with D, soft plastic click. Let me double check by reasoning about the video flow. The diagram with purple highlights appears at 02:20. Then at 02:30-02:50 the person is handling the heatsink and the M.2 area. The locking click of an M.2 heatsink is generally a soft plastic click. Thus answer D.

incorreto

Plano de fundo atrás de um cabo de fonte de alimentação sendo conectado

Modo de falha: Localiza um momento quase idêntico

Pergunta 1539 near-duplicate-moments Qwen3.8 27B

O vídeo completo enviado ao modelo, disponibilizado pelo espelho do OpenRouter usado pelo harness.

Quando a mão insere um cabo de alimentação preto na fonte de alimentação Seasonic na parte traseira do gabinete preto do PC, que superfície está atrás dele?

  1. A)A dark, textured surfacechave
  2. B)A gray carpeted surface
  3. C)A metal mesh surface
  4. D)A plain white surfacemodelo
respondeu D ✗chave A

A pergunta aponta um momento que se repete com pequenas diferenças. O rastreamento se fixa em uma vista traseira entre 15:30 e 15:40 com uma parede branca ao fundo. O elemento-chave é uma superfície escura e texturizada, então o modelo associou a ação a uma tomada semelhante e identificou o fundo a partir dela.

The user wants me to identify the surface behind the action of plugging a black power cable into the Seasonic PSU at the rear of the case. Let me look at the relevant timestamps. Around 15:30-15:40, the video shows the rear of the case with the Seasonic PSU installed, and a hand plugging in a black power cable (the IEC power cord) into the PSU. Looking at the frames around 15:30-15:40, the background behind the case appears to be a plain white wall/surface. There's a white wall with some light switches visible in earlier frames (around 7:50-8:00), and at the rear view (15:30-15:40) the background is a plain white surface. Let me verify by examining the frames. At 15:30, the case is shown from the rear with the Seasonic logo, and the background is white. At 15:40, the hand plugs the cable in, and the background is a plain white surface. So the answer is D) A plain white surface. Let me double check the other options: A) dark textured surface - that's the table surface where components were laid out earlier. B) gray carpeted surface - similar to the table. C) metal mesh surface - that's part of the case itself. The background behind the case at the rear view is white. Thus D.

incorreto

Onde uma garrafa de uísque está quando uma fala é dita

Modo de falha: Nunca localiza a indicação falada

Pergunta 2894 contrastive-hard-negative/transcript_visual Qwen3.8 27B

O vídeo completo enviado ao modelo, disponibilizado pelo espelho do OpenRouter usado pelo harness.

Quando ele diz “Esse tempo todo ele está lá atrás bebendo uísque”, a garrafa de uísque está à esquerda ou à direita do notebook na bancada de Craig?

  1. A)This does not happen in the video.chave
  2. B)To the left of the laptop.modelo
  3. C)To the right of the laptop.
  4. D)Directly behind the laptop.
respondeu B ✗chave A

A pista é uma fala. O rastreamento nunca a encontra, diz que ela “provavelmente ocorre durante a conversa no banco” e responde com base em um quadro entre 60 e 70 segundos. A chave é “não acontece”. Sem a âncora da transcrição, o modelo usou a cena mais parecida no lugar.

The user is asking about a specific moment in the video where someone says "This whole time he's back there drinking whiskey." I need to find this moment and look at the position of the whiskey bottle relative to the laptop on Craig's bench. Let me look at the early part of the video. Around 60-70 seconds, there's a scene with a man sitting at a bench with a laptop, and there's a bottle on the bench. Let me look carefully. At around 60-70 seconds, there's a man in a light blue shirt sitting in a chair at a bench with a laptop. On the bench, to the left of the laptop (from the viewer's perspective), there's a bottle (looks like a whiskey bottle) and a mug. Wait, I need to find the audio line "This whole time he's back there drinking whiskey." This is likely during the scene where two people are at the bench with the laptop. Let me look at the timestamps around 60-130 seconds. At 60-70s: man in blue shirt at bench with laptop, bottle on left side of bench. At 120-130s: close up of laptop showing a 3D model of a table, with a hand typing. The line about drinking whiskey probably occurs during the conversation at the bench. The bottle is on the bench. Looking at the frame at 60-70s, the bottle is to the left of the laptop (viewer's left). The laptop is on the right side of the bench, the bottle and mug are on the left side. So from the viewer's perspective, the whiskey bottle is to the left of the laptop. Thus the answer is B. Let me double check the orientation. The question asks "to the left or right of the laptop on Craig's bench?" In the image at 60-70s, the laptop is on the right portion of the bench, the bottle is on the left portion. So the bottle is to the left of the laptop. Therefore answer B. I'm fairly confident. The line "This whole time he's back there drinking whiskey" matches the scene where one person is at the laptop and another is "back there" drinking. The bottle visible on the bench is to the left of the laptop. Thus B.