TERRA: Learning Transportable Latent Actions through Temporal Effect Representation and Relational Alignment

Key points

• The first is a tension in time: an endpoint difference discards how motion unfolds, while the full sequence admits nuisance variation.1
• TERRA (Temporal Effect Representation and Relational Alignment) describes a transition by a compact temporal effect, its net feature change together with a low-order within-window dynamics component, and learns a continuous latent from this effect.1
• The same effect space then serves as the reference for reuse: Effect-Anchored Transport (EAT) decodes a latent in other initial states and anchors the resulting effect to the one observed at its source, so that the latent is shaped by what it does across contexts rather than only by the transition it came from.1
• With frozen linear readers, TERRA predicts actions more accurately than UniVLA and a LAPA-style baseline, degrades more slowly under visual distractors, and keeps transported transitions faithful to the donor action as the recipient context moves farther away; a same-budget control shows that these gains come largely from EAT.1

Sources (1)

• [1]TERRA: Learning Transportable Latent Actions through Temporal Effect Representation and Relational Alignment
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 06:07 AM

  Latent actions supervise robot policies with action-like codes inferred from visual transitions, and their usefulness hinges on two questions: what a code keeps from a transition, and whether it still means the same thing when reused in a different initial state.

  The first is a tension in time: an endpoint difference discards how motion unfolds, while the full sequence admits nuisance variation.

Extractive summary: sentences quoted from the sources.