Robust Vision-Language Navigation with Awareness of Instruction-Execution Progress
Although instruction-execution progress provides a direct cue for separating executed and pending content, existing progress estimation approaches often rely on explicit progress supervision, sophisticated progress-learning stages, or costly online reasoning with LLMs. In this paper, we propose a robust VLN framework with Awareness of Instruction-Execution Progress (VLN-AIEP), which couples progress estimation with navigation decision-making.
ProofPaper ↗
Key points
- In Vision-Language Navigation (VLN) tasks involving multi-step instructions as guidance, an agent must associate its observations with the relevant instruction content at each decision point.
- This coupling makes correct navigation decisions depend on accurate progress estimation, allowing the progress estimator to learn directly from navigation decision labels without explicit progress annotations.
- To support progress estimation, we construct a landmark-anchored trajectory representation that encodes the semantic labels and viewpoint-relative geometry of instruction-referred landmarks.
- These landmarks serve as spatial references for associating the visited trajectory with executed instruction content.
Sources (1)
- [1]Robust Vision-Language Navigation with Awareness of Instruction-Execution ProgressarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 06:57 AM
Although instruction-execution progress provides a direct cue for separating executed and pending content, existing progress estimation approaches often rely on explicit progress supervision, sophisticated progress-learning stages, or costly online reasoning with LLMs. In this paper, we propose a robust VLN framework with Awareness of Instruction-Execution Progress (VLN-AIEP), which couples progre
In Vision-Language Navigation (VLN) tasks involving multi-step instructions as guidance, an agent must associate its observations with the relevant instruction content at each decision point.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026Qwen/Qwen-Image-2.1-Turbo
- Oct 9, 2026Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding
- Oct 9, 2026Collaborative Theory of Mind: Learning to Self-Revise by Anticipating User Responses
- Oct 9, 2026AMLworld-Compact: Importance-Weighted Downsampling for Cost-Effective LLM Evaluation and Error Diagnosis
- Oct 8, 2026Understanding Jev, the new model everyone is talking about
- Oct 8, 2026Launching an opt-in vulnerability-finding service for open-source software