How Do LLMs Change Predictions Under Negation?

Key points

• Negation is an essential feature of human language, yet large language models (LLMs) remain unreliable in processing it.1
• We evaluate recent open-source and closed-source LLMs on our negation benchmark and find that, in 37-71% of cases, they repeat the same answer under negation (e.g., "Madrid" for "What is not the capital of Spain?").1
• Our main finding is that specialized attention heads and MLP neurons jointly implement negation by (1) suppressing retrieval of the original answer (e.g., "Madrid") while (2) promoting a favored candidate within the answer category (e.g., "Paris").1
• To address this weakness in the model's negation mechanism, we propose a training objective that requires larger shifts in answer preference for more confident original predictions, and show that it reduces negation failures with less degradation of general capabilities than standard fine-tuning baselines.1

Sources (1)

• [1]How Do LLMs Change Predictions Under Negation?
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 07:12 AM

  Furthermore, we find that this difference from human processing is a key source of negation failures: the model's mechanism relies on suppressing the original answer rather than using it to determine what to exclude, so the model can repeat the original answer when suppression is too weak or when a bias toward particular answers prevents it from selecting an alternative.

  Negation is an essential feature of human language, yet large language models (LLMs) remain unreliable in processing it.

Extractive summary: sentences quoted from the sources.