GuidedNav: Shaping Vision-Language Navigation Representations through Pre-action Attention and Spatial Guidance

Key points

• Vision-and-language navigation (VLN) requires an agent to track progress through an extended instruction, identify landmarks relevant to its current objective, and understand the surrounding spatial structure to act.1
• Subtask and landmark guidance steer native pre-action attention toward the active instruction segment and relevant visual regions, grounding action prediction in the current objective.1
• Spatial guidance aligns the policy's internal representations with geometric features that capture long-horizon context and local spatial structure.1
• We also introduce Guided-R2R and Guided-RxR, two annotation-enriched navigation datasets that support more interpretable VLN research.1

Sources (1)

• [1]GuidedNav: Shaping Vision-Language Navigation Representations through Pre-action Attention and Spatial Guidance
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 07:18 AM

  We introduce GuidedNav, a unified framework that explicitly shapes these representations through training-time guidance.

  Vision-and-language navigation (VLN) requires an agent to track progress through an extended instruction, identify landmarks relevant to its current objective, and understand the surrounding spatial structure to act.

Extractive summary: sentences quoted from the sources.