Sparse Planning in Visual World Models via Cost Gradients
We introduce COSTGRAD, a training-free, goal-conditioned selector that ranks spatial tokens by the gradient norm of the planning cost with respect to each input token.
ProofPaper ↗
Key points
- Token-based world models enable fine-grained latent planning, but repeatedly processing large spatial token grids makes action search expensive.
- On AdaLN-conditioned predictors at $50%$ sparsity, COSTGRAD matches or exceeds full-token planning on three of four continuous-control benchmarks, while giving a measured $2.6\times$ wall-clock speedup per environment planning step.
- This difference tracks action-pathway drift: gradient-selected removal produces less drift than random removal on AdaLN, but more on concat.
- These results highlight selector-architecture compatibility as a design axis for sparse world-model planning.
Sources (1)
- [1]Sparse Planning in Visual World Models via Cost GradientsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 03:40 PM
We introduce COSTGRAD, a training-free, goal-conditioned selector that ranks spatial tokens by the gradient norm of the planning cost with respect to each input token.
Token-based world models enable fine-grained latent planning, but repeatedly processing large spatial token grids makes action search expensive.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 7, 2026RoboJEPA: Scaling Laws for Multi-Embodiment Robotic Latent World Models
- Oct 7, 2026DSReg: Provably Recovering Individual World Latents without Reconstruction
- Oct 6, 2026SPW-Nav Streams Language-Guided Panoramic Video in Real Time
- Oct 5, 2026From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation
- Sep 29, 2026Introducing Quine: An AI research system designed for the complexity of biology
- Sep 29, 2026LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation