SimVLA: Zero-Shot Sim-to-Real VLA Learning for Mobile Manipulation

Key points

• Large-scale, diverse datasets have driven the success of LLMs and VLMs. But VLAs for robotics remain limited by the cost and complexity of real-world data collection.1
• While simulation offers a scalable alternative, its potential for sim-to-real VLA learning in mobile manipulation remains largely underexplored.1
• SimVLA is first pre-trained on two complementary simulation-derived datasets: SimAction, a large-scale robot action dataset spanning 35 diverse mobile manipulation tasks, generated by composing atomic skills, and SimVQA, which leverages privileged simulator state to provide spatial, geometric, and subtask-level visual-language supervision.1
• We evaluate SimVLA on tasks including restocking, pouring, and cleaning, and show zero-shot transfer to real-world mobile manipulation, including real home environments.1

Sources (1)

• [1]SimVLA: Zero-Shot Sim-to-Real VLA Learning for Mobile Manipulation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 04:52 AM

  We introduce SimVLA, an end-to-end framework that trains VLAs entirely on synthetic simulation data without teleoperation for mobile manipulation.

  Large-scale, diverse datasets have driven the success of LLMs and VLMs. But VLAs for robotics remain limited by the cost and complexity of real-world data collection.

Extractive summary: sentences quoted from the sources.