WorldGuide: Goal-Directed Video World Model for Procedural Task Execution

Key points

• Video generators and video-based world models can synthesize plausible visual trajectories, but long-horizon procedural tasks require generation to adapt to what has actually been produced.1
• A model must determine the next action from its generated state, execute that action, and recognize when the task is complete.1
• Due to the lack of step-level action-video supervision for joint planner-executor training, we introduce WorldGuide Bench: approximately 59K step-annotated videos across 245 tasks and 27 procedural categories.1
• These results demonstrate the importance of coupling planning with learned execution for goal-directed procedural video generation.1

Sources (2)

• [1]WorldGuide: Goal-Directed Video World Model for Procedural Task Execution
  Hugging Face Daily Papers · Oct 8, 12:00 AM

  We formulate procedural video generation as closed-loop task execution in visual world space and introduce WorldGuide.

  Video generators and video-based world models can synthesize plausible visual trajectories, but long-horizon procedural tasks require generation to adapt to what has actually been produced.

• [2]WorldGuide: Goal-Directed Video World Model for Procedural Task Execution
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:59 PM · same content

Extractive summary: sentences quoted from the sources.