VCR-Bench: A Modular Open-Source Benchmark for Video Classification Robustness
We introduce VCR-Bench, a modular open-source benchmark framework that standardizes video loading, wrappers for classifiers, adversarial attacks and defenses, perceptual metrics, configuration presets, and result logging.
ProofPaper ↗
Key points
- Robustness of image classification has several benchmarks, but their video counterparts are absent.
- In video classification temporal dimension introduces additional degrees of freedom for adversarial attacks, defenses, and preprocessing.
- VCR-Bench currently integrates 30 video classification models, 14 adversarial attacks, and 10 defense wrappers under a common evaluation protocol.
- We evaluate representative video classifiers, attacks, and defenses on Kinetics-400 subset, reporting clean accuracy, attack success rate, perceptual quality, runtime, and memory usage.
Sources (1)
- [1]VCR-Bench: A Modular Open-Source Benchmark for Video Classification RobustnessarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 06:03 PM
We introduce VCR-Bench, a modular open-source benchmark framework that standardizes video loading, wrappers for classifiers, adversarial attacks and defenses, perceptual metrics, configuration presets, and result logging.
Robustness of image classification has several benchmarks, but their video counterparts are absent.
Extractive summary: sentences quoted from the sources.