Selective Listening: Mechanism-Guided Control of Audio Influence in Large Audio-Language Models
Large audio-language models (LALMs) exploit multimodal evidence, yet task-irrelevant audio can alter text-reasoning decisions when listening is unnecessary.
ProofPaper ↗
Key points
- Paired drift analysis and targeted interventions identify architecture-specific, intervention-sensitive late audio pathways as actionable control points.
- We introduce ICAP-Gate, which applies mechanism-guided, task-conditioned control to each model's pathway.
- Fixed suppression degrades automatic speech recognition (ASR) across all four models, whereas ICAP-Gate matches ungated ASR performance by preserving the pathway for explicit audio-demand instructions.
- These results establish selective modality influence control as a design principle for robust multimodal reasoning.
Sources (1)
- [1]Selective Listening: Mechanism-Guided Control of Audio Influence in Large Audio-Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 04:00 AM
Large audio-language models (LALMs) exploit multimodal evidence, yet task-irrelevant audio can alter text-reasoning decisions when listening is unnecessary.
Paired drift analysis and targeted interventions identify architecture-specific, intervention-sensitive late audio pathways as actionable control points.
Extractive summary: sentences quoted from the sources.