PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation
To address these challenges, we present PhysTacGen, a visual-to-optical-tactile image generation framework that integrates material-aware descriptions with geometric conditioning.
Key points
- Realistic physical interaction is a cornerstone of embodied intelligence, yet collecting paired visual--tactile data remains costly.
- Visual-to-tactile synthesis offers a promising approach to augmenting such data, but learning this mapping is complicated by the gap between visual appearance and contact-related material properties, as well as spatial misalignment in paired observations.
- First, we introduce Group Tactile Policy Optimization (GTPO), a reinforcement learning strategy that refines a vision--language model to generate structured material descriptions using task-specific rewards.
- Generated tactile inputs also improve performance on an attribute-derived force-coefficient prediction proxy.
Sources (1)
- [1]PhysTacGen: Physics-Aware Visual-Tactile Sensor Image GenerationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 10:00 AM
To address these challenges, we present PhysTacGen, a visual-to-optical-tactile image generation framework that integrates material-aware descriptions with geometric conditioning.
Realistic physical interaction is a cornerstone of embodied intelligence, yet collecting paired visual--tactile data remains costly.
Extractive summary: sentences quoted from the sources.