A Lightweight Velocity Correction Framework for Improving Instruction Following in Few-Step Text-to-Image Generation

Key points

• We propose a lightweight correction framework for a frozen 6B-parameter generator.1
• The generator predicts velocities that determine how its intermediate image representation is updated; a corrector with fewer than 20M parameters learns residual adjustments at selected sampling steps.1
• To our knowledge, this is the first framework for lightweight velocity-residual adaptation of fully frozen few-step text-to-image generators that preserves their original sampling schedule and forward-pass count.1
• On our target complex-scene distribution, the framework achieves a +17.60-percentage-point NetWin over frozen Z-Image-Turbo in pairwise instruction-following evaluation, with only 26.9 ms (0.84%) additional end-to-end latency per image and no increase in backbone evaluations.1

Sources (1)

• [1]A Lightweight Velocity Correction Framework for Improving Instruction Following in Few-Step Text-to-Image Generation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 02:43 AM

  Few-step text-to-image generators produce high-quality images with few network forward passes, yet struggle with complex prompts involving multiple subjects, interactions, spatial relations, counting, and attribute binding.

  We propose a lightweight correction framework for a frozen 6B-parameter generator.

Extractive summary: sentences quoted from the sources.