Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?

Key points

• Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective.1
• However, as we show through extensive analysis, sufficiently accurate action reconstruction is only one part of what makes a downstream robot policy successful.1
• In this work, we introduce Predictable and Robust Action Tokenization (ProAct), a tokenizer training method that strategically augments reconstruction with the goal of improving downstream predictability and robustness.1
• These results suggest that effective action tokenization should be designed as a policy interface that balances fidelity, predictability, and robustness, rather than as a reconstruction problem alone.1

Sources (1)

• [1]Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 10:12 PM

  Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space.

  Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective.

Extractive summary: sentences quoted from the sources.