From Surface to Depth: Towards Cognitive Appraisal Reasoning in Multimodal Emotion Understanding

Key points

• Recent multimodal large language models (MLLMs) increasingly incorporate explainable reasoning for emotion understanding.1
• However, reasoning based mainly on observable affective cues can reduce emotion understanding to superficial cue-label associations, giving rise to the Clever Hans effect.1
• In contrast, human emotions are shaped by how individuals interpret and evaluate surrounding events beyond observable cues.1
• CogEmo-40K is a large-scale instruction-tuning dataset constructed through a perception-to-appraisal pipeline to elicit evidence-grounded reasoning across six cognitive appraisal dimensions underlying emotion.1

Sources (1)

• [1]From Surface to Depth: Towards Cognitive Appraisal Reasoning in Multimodal Emotion Understanding
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 01:14 PM

  Inspired by appraisal theories of emotion, we formulate multimodal emotion understanding as a progression from perception to cognitive appraisal, and introduce a dataset, a model, and a benchmark to support this novel paradigm.

  Recent multimodal large language models (MLLMs) increasingly incorporate explainable reasoning for emotion understanding.

Extractive summary: sentences quoted from the sources.