Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving

Key points

• Offline reinforcement learning enables reward-driven policy improvement from fixed datasets without requiring online exploration, making it particularly attractive in safety-critical domains.1
• In interactive environments such as autonomous driving, this can be insufficient: a candidate ego trajectory may remain well supported under the marginal behavior distribution while being poorly supported jointly with the surrounding-agent behavior observed in the logged interaction.1
• Starting from the joint data distribution over ego and surrounding-agent futures, we show that joint-support degradation decomposes exactly into an ego-support component and a residual interaction-support component.1
• Closed-loop evaluations on nuPlan, Interplan and real-world truck experiments show that ICDP suppresses high-value yet interaction-unsupported trajectory selections and improves performance in interaction-critical driving scenarios.1

Sources (1)

• [1]Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 09:48 AM

  We refer to this degradation in interaction support as interaction distribution shift (IDS), and introduce Interaction-Constrained Drive Policy (ICDP), an offline reinforcement learning framework that explicitly controls interaction-level distribution shift.

  Offline reinforcement learning enables reward-driven policy improvement from fixed datasets without requiring online exploration, making it particularly attractive in safety-critical domains.

Extractive summary: sentences quoted from the sources.