One for All, All for One: Coordinated Multi-Agent Diffusion Steering via Stochastic Optimal Control

Key points

• Deep generative models often produce structured outputs composed of interacting components.1
• We pursue a modular alternative: reuse independently trained component generators and learn only how to coordinate them to produce coherent structured outputs.1
• We formulate coordination as a stochastic optimal control problem, balancing an assembly-level reward that specifies the desired properties of the combined output against deviations from the pretrained dynamics.1
• Across multi-agent maze navigation, articulated robot planning, and text-conditioned human motion, CMDS turns frozen models into coordinated multi-agent generators.1

Sources (1)

• [1]One for All, All for One: Coordinated Multi-Agent Diffusion Steering via Stochastic Optimal Control
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 04:02 PM

  Our framework, Coordinated Multi-Agent Diffusion Steering (CMDS), treats frozen pretrained diffusion models as reusable generative primitives and coordinates their reverse processes through a learned control.

  Deep generative models often produce structured outputs composed of interacting components.

Extractive summary: sentences quoted from the sources.