Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes
We propose TRACE, a cognition-oriented framework that formalizes an affective episode through three interrelated stages: Condition, Affect, and Effect, integrating observable cues with cognitive factors such as internal stance and regulation of emotional display.
ProofPaper ↗
Key points
- Affective computing has progressed from categorical emotion recognition to open-ended affective analysis with large multimodal models.
- Yet affective science describes emotion as an unfolding process shaped by appraisal, regulation, and social interpretation, which remains underexplored computationally.
- Based on this formulation, TRACE-Bench evaluates multimodal models in real-world social scenes through five tasks spanning grounded affect recognition, regulation decoding, cause reasoning, effect reasoning, and full-chain reconstruction, with 3,746 structured question-answer pairs over 646 videos.
- A matched human-model comparison reveals a substantial performance gap, while affect-specialized models also generally lag behind general-purpose MLLMs. Model outputs show recurring failures, including treating displayed behavior as genuine feeling and fabricating unsupported events during long-chain generation.
Sources (1)
- [1]Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social ScenesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 07:40 AM
We propose TRACE, a cognition-oriented framework that formalizes an affective episode through three interrelated stages: Condition, Affect, and Effect, integrating observable cues with cognitive factors such as internal stance and regulation of emotional display.
Affective computing has progressed from categorical emotion recognition to open-ended affective analysis with large multimodal models.
Extractive summary: sentences quoted from the sources.