When to Unpair: Regulating Pairing Dependence in Medical Visual In-Context Learning

Key points

• We diagnose dependence on individual pairings with a test-time derangement that reassigns every support label to another support image while preserving the query, support images, and label multiset.1
• Further analysis of a paired-trained model reveals support-associated spurious regions and lesion-size biases even with real, unaltered supports, alongside sensitivity to mis-registered support labels.1
• To regulate this dependence, we introduce a late unpairing curriculum (LUC), which starts with matched training and then applies random unpairing, replacing each support label with that of another support in the same episode.1
• In a released model, brief fine-tuning with random unpairing reduces the gap.1

Sources (1)

• [1]When to Unpair: Regulating Pairing Dependence in Medical Visual In-Context Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 04:17 PM

  Visual in-context learning (ICL), well suited to label-scarce medical imaging, uses support image-label pairs to demonstrate input-output mappings, while the labels collectively indicate the requested task.

  We diagnose dependence on individual pairings with a test-time derangement that reassigns every support label to another support image while preserving the query, support images, and label multiset.

Extractive summary: sentences quoted from the sources.