ΔWAM: Distilling Action Tangent Fields into World Action Models

Key points

• However, much of the predictable future is dominated by appearance and scene persistence rather than action-dependent dynamics.1
• We observe that several recent WAM designs, including optical flow, motion-centric representations, and latent actions, can be understood from a common perspective in which world supervision becomes more efficient as it contains a higher proportion of action-relevant variation.1
• Based on this insight, we introduce Action Tangent Fields, which reformulate world supervision through a local Taylor expansion of how actions induce changes in future dynamics.1
• We represent future dynamics in Residual-VAE space, where the future latent remains recoverable from the current latent and its residual, and use a strong action-conditioned world model (ACWM) to probe the local correspondence between action variations and residual-world variations.1

Sources (1)

• [1]ΔWAM: Distilling Action Tangent Fields into World Action Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 09:28 AM

  World Action Models (WAM) improve robot policies by augmenting sparse action supervision with dense future prediction.

  However, much of the predictable future is dominated by appearance and scene persistence rather than action-dependent dynamics.

Extractive summary: sentences quoted from the sources.