DataVista: Diagnosing Multimodal LLMs on Data Video Understanding
We present DataVista, the first benchmark for data video understanding, containing 961 real-world data videos and 6,775 evaluation questions organized under a three-level progressive capability framework (data perception, temporal reasoning, narrative understanding) with 10 fine-grained question types across five topic domains.
Key points
- Data video is a media form that integrates data visualization with video narrative, widely adopted in news reporting and business analysis.
- Compared with general video understanding, data video understanding places greater emphasis on accurately reading data from animated charts, integrating evidence across charts and time, and understanding how narrative organization and visual design communicate information.
- Yet existing benchmarks target either general videos or static charts, and data video understanding has not been systematically evaluated.
- Further analysis of model responses identifies typical failure modes in chart reading, evidence judgment, and instruction understanding.
Sources (1)
- [1]DataVista: Diagnosing Multimodal LLMs on Data Video UnderstandingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 02:02 PM
We present DataVista, the first benchmark for data video understanding, containing 961 real-world data videos and 6,775 evaluation questions organized under a three-level progressive capability framework (data perception, temporal reasoning, narrative understanding) with 10 fine-grained question types across five topic domains.
Data video is a media form that integrates data visualization with video narrative, widely adopted in news reporting and business analysis.
Extractive summary: sentences quoted from the sources.