From 2.1M Hours of Raw Broadcast Audio to Finnish Speech SSL: Confidence Filtering and Architectural Analysis at Scale
We propose automatic speech recognition (ASR) confidence as a proxy for pretraining data quality and show 3-7% relative WER improvements over LID-based filtering despite a smaller training set.
ProofPaper ↗
Key points
- Broadcast archives offer scale and diversity for self-supervised learning (SSL) but contain substantial noisy, non-speech, and multilingual content.
- Whether and how to filter unlabeled speech by quality for SSL, particularly from noisy broadcasts, remains underexplored.
- We present a large-scale Finnish SSL study starting from 2.1M hours of unconstrained broadcast audio.
- We further address the lack of controlled architectural comparisons in speech SSL by evaluating Transformer, Macaron-Net, Conformer, and E-Branchformer at matched parameter budgets.
Sources (1)
- [1]From 2.1M Hours of Raw Broadcast Audio to Finnish Speech SSL: Confidence Filtering and Architectural Analysis at ScalearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 06:57 AM
We propose automatic speech recognition (ASR) confidence as a proxy for pretraining data quality and show 3-7% relative WER improvements over LID-based filtering despite a smaller training set.
Broadcast archives offer scale and diversity for self-supervised learning (SSL) but contain substantial noisy, non-speech, and multilingual content.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026Qwen/Qwen-Image-2.1-Turbo
- Oct 9, 2026Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash
- Oct 9, 2026Learning Representations to Guide Pixel Diffusion Transformers
- Oct 8, 2026LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation
- Oct 8, 2026One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
- Oct 8, 2026Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals