The Confidence Game: Strategic Miscalibration in Human-AI Delegation

Key points

• However, when agents seek to maximize user engagement or revenue, confidence reports may be strategically distorted, detracting from their informativeness.1
• We formalize this problem in the Confidence Game: a repeated signaling game with imperfect monitoring in which an agent of unknown honesty and ability reports its confidence, and a user decides whether to delegate the task or complete it herself.1
• We characterize the Markov Perfect Bayesian Equilibria of the two-period game and show that honest reporting is not an equilibrium, inflation is the unique best response once the agent is sufficiently myopic, and under-reporting requires that the user believe honesty to be a minority.1
• Furthermore, we find that the LLM agent's decisions are coherent, but it systematically underestimates both how likely the user is to delegate and how secure its reputation is, resulting in less extreme behavior.1

Sources (1)

• [1]The Confidence Game: Strategic Miscalibration in Human-AI Delegation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 03:25 AM

  Calibrated uncertainty quantification is essential to ensuring AI agents are trustworthy and reliable.

  However, when agents seek to maximize user engagement or revenue, confidence reports may be strategically distorted, detracting from their informativeness.

Extractive summary: sentences quoted from the sources.