AudioTETRIS: Token Elimination via Transitional Importance Signals for Efficient Audio LLMs

Key points

• Large Audio Language Models (LALMs) achieve strong performance on speech-text tasks, but standard audio encoders produce hundreds of tokens for even short audio clips.1
• Current training-free token compression methods prune or merge tokens based solely on coarse attention or feature similarity.1
• However, treating audio tokens as generic, isotropic features smooths over spectral transitions, destroying fine-grained phonetic transitions essential for decoding speech.1
• Evaluated on Qwen2-Audio-7B, Phi-4-Multimodal, and Audio Flamingo 3, AudioTETRIS retains near-uncompressed performance on several speech tasks using only 25% of audio tokens, matches or surpasses full-token accuracy at moderate compression rates, and consistently outperforms existing training-free baselines.1

Sources (1)

• [1]AudioTETRIS: Token Elimination via Transitional Importance Signals for Efficient Audio LLMs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 01:49 PM

  We propose AudioTETRIS, a training-free, two-stage framework that uses the audio encoder's representation geometry as an acoustic prior for token importance before applying attention-based compression.

  Large Audio Language Models (LALMs) achieve strong performance on speech-text tasks, but standard audio encoders produce hundreds of tokens for even short audio clips.

Extractive summary: sentences quoted from the sources.