DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models
We introduce DiffusionART (Diffusion Reinforcement with Aggregation-First Reward Tilting), which aggregates each prompt's rewards into a shared reward-improving direction for reverse-process states, removing the consistency force by construction, and adds a trust-region-masked second update for drift-adaptive restoration without EMA tuning.
ProofPaper ↗
Key points
- Online reinforcement learning (RL) fine-tunes diffusion models with black-box rewards, but reverse-process methods such as Flow-GRPO are limited in efficiency by poor credit assignment, while forward-process methods such as DiffusionNFT are efficient yet unstable without carefully tuned exponential-moving-average (EMA) off-policy training.
- We show that NFT-style objectives contain a hidden forward--reverse consistency force that joint positive--negative training cancels only partially, and that EMA stabilizes training through restoration rather than off-policy rollouts.
- Across image and video generation, DiffusionART trains efficiently and stably without per-task tuning.
- In particular, on HunyuanVideo-13B with equally weighted visual and motion quality rewards, DiffusionART reaches the late-stage mean reward levels of DiffusionNFT and Flow-GRPO-Fast with 12.02 and 11.72 times fewer GPU hours, respectively, while achieving 28.8% and 53.0% higher mean training rewards.
Sources (2)
- [1]DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 06:01 PM
We introduce DiffusionART (Diffusion Reinforcement with Aggregation-First Reward Tilting), which aggregates each prompt's rewards into a shared reward-improving direction for reverse-process states, removing the consistency force by construction, and adds a trust-region-masked second update for drift-adaptive restoration without EMA tuning.
Online reinforcement learning (RL) fine-tunes diffusion models with black-box rewards, but reverse-process methods such as Flow-GRPO are limited in efficiency by poor credit assignment, while forward-process methods such as DiffusionNFT are efficient yet unstable without carefully tuned exponential-moving-average (EMA) off-policy training.
- [2]DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion ModelsHugging Face Daily Papers · Oct 8, 12:00 AM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 7, 2026BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation
- Oct 7, 2026Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression
- Oct 7, 2026World Potential Model: Pretrained World Knowledge as Progress Potentials
- Oct 7, 2026Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning
- Oct 6, 2026Personalize at Test Time: Learning User Preferences for Image Generation
- Oct 6, 2026Verify Less, Evolve More: Training Idea-Level Critics for Verification-Efficient ML Evolving Agents