EgoPhys: Estimating Peak Contact Force and Mechanical Work from Egocentric Manipulation Video

Key points

• Peak contact force and mechanical work quantify these complementary aspects, but estimating them from egocentric video is challenging because physical interaction cues are local and indirect.1
• Moreover, peak force is associated with brief contact events, whereas mechanical work depends on force-motion coupling throughout the contact duration.1
• To address these challenges, we propose EgoPhys, an RGB-only framework comprising Contact-Aware Spatial Aggregation (CASA) and Target-Specific Multi-Expert Temporal Routing (TMTR).1
• On the test split from Hoi! dataset, EgoPhys substantially improves predictions of peak force and mechanical work, achieving MAEs of \(5.205 \pm 0.584\) $N$ and $0.894 \pm 0.081$ $J$, respectively.1

Sources (1)

• [1]EgoPhys: Estimating Peak Contact Force and Mechanical Work from Egocentric Manipulation Video
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 06:41 AM

  Physically grounded manipulation of articulated objects requires understanding both the maximum forces encountered during contact and the work performed as their parts move.

  Peak contact force and mechanical work quantify these complementary aspects, but estimating them from egocentric video is challenging because physical interaction cues are local and indirect.

Extractive summary: sentences quoted from the sources.