Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict

Key points

• When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion?1
• Through knowledge conflict, situations where the backbone language model's parametric knowledge contradicts the evidence it encounters, or where two contextual sources disagree, we evaluate two conflict settings: (1) controlled conflict and (2) naturally occurring conflict during multi-step agentic execution, each paired with matched no-conflict controls.1
• Evaluating four agents, we find that higher task accuracy does not necessarily correspond to greater epistemic humility: some high-accuracy configurations recognize conflicts during execution but do not communicate unresolved uncertainty in their incorrect final answers.1
• Finally, we show that model-level interventions can improve EH, but often at the cost of task accuracy, suggesting that epistemic humility emerges from the interaction among the backbone model, agent harness, and evaluation environment.1

Sources (2)

• [1]Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
  Hugging Face Daily Papers · Oct 8, 12:00 AM

  We propose to evaluate agents on epistemic humility (EH): the agent's willingness to recognize, act on, and communicate uncertainty during task execution.

  When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion?

• [2]Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:25 PM · same content

Extractive summary: sentences quoted from the sources.