Streaming-Aware Diffusion for Real-Time Video Super-Resolution via Cross-Step Attention

Key points

• Real-time video super-resolution requires high spatio-temporal fidelity under strict latency constraints, challenging diffusion models due to their iterative sampling cost and limited temporal coordination.1
• Our Cross-Step Attention mechanism reuses intermediate denoising features across adjacent frames and diffusion steps, enabling temporal information exchange without explicit temporal modeling.1
• We further introduce Trajectory-Coupled Diffusion Scheduling, which aligns adjacent diffusion states and provides cleaner intermediate representations for cross-step conditioning, improving temporal coherence.1
• These components are integrated into a streaming inference pipeline that incrementally propagates latent states across frames, reducing the effective computational complexity from $O(N \cdot S)$ to $O(N + S)$ for $N$ frames and $S$ diffusion steps.1

Sources (1)

• [1]Streaming-Aware Diffusion for Real-Time Video Super-Resolution via Cross-Step Attention
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 11:35 AM

  We propose a streaming-aware framework that adapts pretrained single-image latent diffusion models for efficient video super-resolution (VSR) by exploiting the sequential structure of video streams.

  Real-time video super-resolution requires high spatio-temporal fidelity under strict latency constraints, challenging diffusion models due to their iterative sampling cost and limited temporal coordination.

Extractive summary: sentences quoted from the sources.