MIM-VLA: Learning Physical Interaction Representations from Gripper Motor Feedback

Key points

• Vision-language-action (VLA) policies infer grasp actions primarily from visual observations and robot state, but do not explicitly represent the physical response observed after contact.1
• A motor-only Motor Interaction Module (MIM) is pretrained with human-reviewed contact and interaction-phase labels and then conditions only the gripper-action pathway of SmolVLA; arm actions and the position-control interface remain unchanged.1
• The same token supports the MEM selector VLM that compares candidate interactions and produces evidence-conditioned selections and explanations.1
• We evaluate MIM-VLA in three real-world settings: comparing the interaction resistance of visually different objects, disambiguating visually similar real and replica objects through active probing, and gently grasping fragile objects, including held-out instances.1

Sources (1)

• [1]MIM-VLA: Learning Physical Interaction Representations from Gripper Motor Feedback
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 02:28 PM

  We present MIM-VLA, a motor-feedback-based architecture that encodes recent gripper current, position, velocity, and signal validity as a 128-dimensional interaction token.

  Vision-language-action (VLA) policies infer grasp actions primarily from visual observations and robot state, but do not explicitly represent the physical response observed after contact.

Extractive summary: sentences quoted from the sources.