LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL

Key points

• While offline reinforcement learning (RL) enables policy optimization from static datasets without costly online interaction, it remains bottlenecked by the risk of executing out-of-distribution (OOD) actions.1
• Recent approaches mitigate this by learning a behavior-cloning policy through flow matching and then performing RL within its constrained latent space.1
• In this work, we find that entropy regularization is essential in latent-space RL for addressing these challenges.1
• Through comprehensive experiments on 40 challenging OGBench tasks with varying dataset qualities, we show that LASER achieves state-of-the-art performance.1

Sources (1)

• [1]LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 06:47 PM

  We introduce LASER, a novel offline RL algorithm that applies latent-space adjoint matching to achieve entropy-regularized latent-space RL with expressive flow policies while avoiding backpropagation through time.

  While offline reinforcement learning (RL) enables policy optimization from static datasets without costly online interaction, it remains bottlenecked by the risk of executing out-of-distribution (OOD) actions.

Extractive summary: sentences quoted from the sources.