Compact Robot Policies Need Fine-Grained Visual Representations

Key points

• Multi-task manipulation policies differ in architecture, scale, and pretrained priors all at once, so published comparisons cannot attribute performance to any single component.1
• We argue that most of it comes from the visual representation, and that parameter scale and generative priors are largely incidental.1
• Compression matters as much: resampling each view to 48 tokens beats passing all patch tokens (97.0% vs 83.2%), and a variational information bottleneck over those tokens is worse than a hard token budget, cutting LIBERO-Goal from 95.8% to 33.0% by suppressing the instruction-dependent token selection the policy relies on.1
• Therefore, we argue that a compact policy works when its representation is pretrained, task-adapted, and compressed.1

Sources (1)

• [1]Compact Robot Policies Need Fine-Grained Visual Representations
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 11:36 AM

  To test this, we build CoRP (Compressed Representation Policy), a deliberately compact policy (48.9M parameters, no vision-language model and no video-generative prior) that factorizes into a representation extractor and a flow-matching action generator.

  Multi-task manipulation policies differ in architecture, scale, and pretrained priors all at once, so published comparisons cannot attribute performance to any single component.

Extractive summary: sentences quoted from the sources.