Factorized Tactile Representation and Control for Sim-to-Real Manipulation

Key points

• Tactile sim-to-real learning must bridge simulated contact and device-specific sensor responses while preserving information needed for control.1
• The response is separated into contact geometry, force distribution, and temporal contact change, with representation-specific encoding and randomization.1
• We evaluate the approach through response reconstruction, spatial alignment, force regulation, and contact-rich adversarial peg insertion in simulation and the real world, enabling the utility and transfer reliability of different tactile representations to be assessed independently.1
• The approach achieves <1 mm contact localization, 1.69 N force-tracking error on unseen geometries, and a 35% improvement in real-world adversarial peg insertion over the unfactorized response, with different tactile representations benefiting different interactions.1

Sources (1)

• [1]Factorized Tactile Representation and Control for Sim-to-Real Manipulation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 05:51 PM

  We propose a factorized tactile representation and control framework that maps normal force and contact patch to an effective contact response recoverable from sensor readings.

  Tactile sim-to-real learning must bridge simulated contact and device-specific sensor responses while preserving information needed for control.

Extractive summary: sentences quoted from the sources.