SPW-Nav Streams Language-Guided Panoramic Video in Real Time

Key points

• The system translates instructions into camera motion using spherical rotation decoupling and pose-aligned conditioning to keep translation inputs bounded over long streams.1
• A multi-term memory and a few-step generator enable scene continuation while supporting on-the-fly instruction switching.1
• The authors created SPW-NavSet, comprising panoramic videos paired with camera trajectories and verified instructions.1
• SPW-Nav achieves higher video quality and camera-following accuracy than prior panoramic generators.1

Why it matters

• By enabling real-time, instruction-driven panoramic streaming from a single image, SPW-Nav aids interactive 3D exploration, virtual reality, and embodied agent training.1

Sources (2)

• [1]SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation
  Hugging Face Daily Papers · Oct 6, 12:00 AM

  Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands m

  SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories

• [2]SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 06:05 PM · same content

AI summary (gemini-3.8-flash); 100% of sentences verified against the sources.