Stable Scores, Unstable Answers: Frame Phase and Option Order in Video Multiple-Choice Evaluation

Key points

• The grid has two parameters, a rate and a phase, and benchmarks report only the rate.1
• The phase moves answers: two deployed samplers differing only by a half-step phase offset answer 23.6% of questions differently while scoring within a point, and across four releases from two families shifting only the phase changes roughly one answer in five after controlling option order.1
• PHASEFUSION decodes three offset grids and averages the option posteriors.1
• Option order, which changes only the presentation, is flagged instead by a one-pass answer margin.1

Sources (1)

• [1]Stable Scores, Unstable Answers: Frame Phase and Option Order in Video Multiple-Choice Evaluation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 04:36 PM

  Video-language models are ranked by multiple-choice accuracy on frames from a uniform grid.

  The grid has two parameters, a rate and a phase, and benchmarks report only the rate.

Extractive summary: sentences quoted from the sources.