When History Helps and Hurts: Selective History Use across Multimodal Turns
Reliable multimodal interaction depends on selective use of conversational history: an earlier question may remain relevant while its previous answer is outdated, whereas a current request may depend on historical evidence despite conflicting new observations.
ProofPaper ↗
Key points
- Existing multi-turn evaluations rarely separate these history-use demands from underlying question difficulty.
- To address this gap, we introduce ReTurn, a benchmark of 7,000 base tasks spanning visual and audio evidence for evaluating selective history use.
- For task-carrying history, Reconfirm/Reground require applying a historical question to current media while varying historical agreement; for evidence-carrying history, Retrieve/Rebind require answering a current question using historical media while varying current-media competition.
- Behavioral probes show that high question recall can coexist with weaker task application, while competing media can redirect answers away from historical targets.
Sources (1)
- [1]When History Helps and Hurts: Selective History Use across Multimodal TurnsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 01:36 PM
Reliable multimodal interaction depends on selective use of conversational history: an earlier question may remain relevant while its previous answer is outdated, whereas a current request may depend on historical evidence despite conflicting new observations.
Existing multi-turn evaluations rarely separate these history-use demands from underlying question difficulty.
Extractive summary: sentences quoted from the sources.