The Deceptive Bandit Problem: Exploratory Coupling and the Fragility of Multi-Agent Learning

Key points

• Randomized exploration is central to bandit learning, multi-agent reinforcement learning, and zeroth-order policy search, yet its independence and privacy are usually only treated as technical assumptions.1
• We show that these properties are critical for security purposes and demonstrate how an adversarial agent can exploit privileged information on another agent's exploration.1
• We analyze a deceiver-victim pair in the minimal two-player strongly monotone setting, where a deceptive player obtains leaked signals that are merely correlated with the victim's exploration.1
• We prove that the deceptive bandit learning (DBL) dynamics converge to an arbitrarily small neighborhood of the DNE while retaining optimal convergence rates.1

Sources (1)

• [1]The Deceptive Bandit Problem: Exploratory Coupling and the Fragility of Multi-Agent Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 09:12 PM

  We show that, by coupling their own exploratory action with this information, the deceptive player injects an externality that steers the learning dynamics to a new steady state, called the deceptive Nash equilibrium (DNE).

  Randomized exploration is central to bandit learning, multi-agent reinforcement learning, and zeroth-order policy search, yet its independence and privacy are usually only treated as technical assumptions.

Extractive summary: sentences quoted from the sources.