MiniWAM: Learning Compact Future Targets for Efficient World-Action Modeling
To construct these targets, we propose Predictive Representations via Inverse Spatiotemporal Modeling (PRISM), which combines inverse-dynamics supervision with feature reconstruction to emphasize control-relevant transition information while preserving useful future-state information.
ProofPaper ↗
Key points
- World modeling has emerged as an effective co-training objective for robot policies, giving rise to World Action Models (WAMs) that jointly predict actions and future states.
- We introduce MiniWAM, which instead predicts compact future representations learned from privileged current-future transitions.
- With the learned PRISM encoder frozen, MiniWAM is trained to jointly predict the resulting targets and robot actions from current observations.
- These results demonstrate that effective world-action modeling does not require predicting native visual futures, and that compact predictive representations provide a strong and substantially more efficient target for policy learning.
Sources (1)
- [1]MiniWAM: Learning Compact Future Targets for Efficient World-Action ModelingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 03:51 PM
To construct these targets, we propose Predictive Representations via Inverse Spatiotemporal Modeling (PRISM), which combines inverse-dynamics supervision with feature reconstruction to emphasize control-relevant transition information while preserving useful future-state information.
World modeling has emerged as an effective co-training objective for robot policies, giving rise to World Action Models (WAMs) that jointly predict actions and future states.
Extractive summary: sentences quoted from the sources.