FearCaut-Qwen: Affective Steering in a Vision-Language Model Shifts the Decision Criterion for Hazard Assessment

Key points

• This study examines that deficiency by using signal detection theory to decompose the decision behavior into perceptual capability and decision-criterion placement.1
• We then propose a novel method for correcting the over-conservative decision policy, inspired by the finding that fear makes humans risk-averse, and ask whether an affective representation associated with fear can be causally manipulated to similarly alter a VLM's decision tendency.1
• Using mechanistic interpretability, we localize a causally implicated affective circuit in the model and use activation steering to manipulate it while observing the effect on downstream prediction.1
• An affective direction is localized on emotion-rich natural scenes, causally validated by sparse-neuron knockout and distributed steering on held-out emotion data, and then injected into the building task.1

Sources (1)

• [1]FearCaut-Qwen: Affective Steering in a Vision-Language Model Shifts the Decision Criterion for Hazard Assessment
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 01:57 PM

  Vision-language models (VLMs) show great potential for damage assessment after a disaster, but a recurring deficiency is that they are reluctant to declare a hazard; that is, recall is low even when overall accuracy appears adequate.

  This study examines that deficiency by using signal detection theory to decompose the decision behavior into perceptual capability and decision-criterion placement.

Extractive summary: sentences quoted from the sources.