Diagnosing and Recovering from Observation-Space Shift at Long-Horizon Skill Seams
Using privileged simulator resets, we find that the dominant shift comes from displaced scene state (e.g., an open drawer or secondary objects left behind by earlier skills), not from the robot's joint configuration or the object the downstream skill manipulates.
Key points
- Long-horizon robotic manipulation is often built by chaining independently trained skills.
- We study this failure mode, Observation-Space Shift (OSS), and ask what causes these skill-seam failures.
- To test this diagnosis, we build a fully learned detect-restore-resume system: a task-progress monitor detects the stall, a learned policy restores the displaced scene components, and seam-robust fine-tuning lets the skill resume.
- On a real Franka arm running a fine-tuned $π{0.5}$ policy, the same monitor is limited by exterior-camera observability, yet closing the loop still recovers some otherwise-terminal failures, motivating wrist and gripper sensing.
Sources (1)
- [1]Diagnosing and Recovering from Observation-Space Shift at Long-Horizon Skill SeamsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 07:15 PM
Using privileged simulator resets, we find that the dominant shift comes from displaced scene state (e.g., an open drawer or secondary objects left behind by earlier skills), not from the robot's joint configuration or the object the downstream skill manipulates.
Long-horizon robotic manipulation is often built by chaining independently trained skills.
Extractive summary: sentences quoted from the sources.