Dialect-Robust Speech Language Models with Synthetic Pseudo-Dialect Augmentation
We propose synthesizing pseudo-dialect speech by converting LLM-generated dialect text via a standard-language TTS model, requiring zero real dialect speech.
ProofPaper ↗
Key points
- Speech Language Model (SLM) performance often degrades on dialects due to data scarcity.
- Conventional text-to-speech (TTS) augmentation struggles to cover diverse dialects as it requires a certain amount of real dialect speech.
- Additionally, we introduce intermediate standard-text prediction during training, acting as semantic normalization for downstream tasks.
- We evaluate dialect understanding via dialect-to-English speech translation across Japanese, German, and Chinese dialects.
Sources (1)
- [1]Dialect-Robust Speech Language Models with Synthetic Pseudo-Dialect AugmentationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 02:25 AM
We propose synthesizing pseudo-dialect speech by converting LLM-generated dialect text via a standard-language TTS model, requiring zero real dialect speech.
Speech Language Model (SLM) performance often degrades on dialects due to data scarcity.
Extractive summary: sentences quoted from the sources.