Incidental information contaminates patient notes and disrupts clinical reasoning in large language models

Key points

• Large language models (LLMs) are increasingly relied upon to support ambient documentation and clinical reasoning.1
• Here we examine the impact of a failure mode shared between these two applications by assessing their sensitivity to information incidental to the patient encounter.1
• In 576 patient-clinician dialogues, we found that frontier models inserted small-talk exchanges into 35% of notes, while mean quality scores changed by at most 0.20 points on five-point scales.1
• These findings support evaluating resistance to incidental information before clinical use, with safeguards that prevent contamination while preserving clinical reasoning.1

Sources (2)

• [1]Incidental information contaminates patient notes and disrupts clinical reasoning in large language models
  Hugging Face Daily Papers · Oct 6, 12:00 AM

  We propose a dual encoding hypothesis of clinical reasoning and distraction in LLMs, with preliminary evidence that LLM components associated with disruption by incidental information also support clinical reasoning.

  Large language models (LLMs) are increasingly relied upon to support ambient documentation and clinical reasoning.

• [2]Large language models are vulnerable to incidental information in clinical documentation and reasoning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 03:55 PM · same content

Extractive summary: sentences quoted from the sources.