Video Codec-Guided Training-Free Token Compression for Video Large Language Models

Key points

• Existing training-free compression methods mainly reduce the number of tokens sent to the language model, but the compression process itself can still require many visual feature comparisons, especially for intra-frame token protection and inter-frame temporal token merging.1
• This paper proposes a video codec-guided training-free token compression method for VLLMs that uses coding information already available in compressed video as a low-cost search prior.1
• Experiments on three representative VLLMs and four video-understanding benchmarks show that the proposed method achieves performance comparable to the state-of-the-art training-free baseline FlashVID, and in some settings surpasses it, while substantially reducing the feature comparisons required by token protection and temporal matching.1
• These results demonstrate that codec information can effectively improve the accuracy-complexity tradeoff of video token compression.1

Sources (1)

• [1]Video Codec-Guided Training-Free Token Compression for Video Large Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 03:56 AM

  Video large language models (VLLMs) rely on long visual token sequences to understand complex temporal content, which makes visual token compression important for efficient inference.

  Existing training-free compression methods mainly reduce the number of tokens sent to the language model, but the compression process itself can still require many visual feature comparisons, especially for intra-frame token protection and inter-frame temporal token merging.

Extractive summary: sentences quoted from the sources.