Geometry-Supervised Visual Representation Learning for Multi-Phenotype Lesion Interpretation in Medical VLMs
To address these limitations, we propose PureVision, a geometry-supervised visual representation learning framework for multi-phenotype lesion interpretation in medical VLMs. It combines a geometry-supervised representation learning module, PureEyes, and an anatomy-guided evidence aggregation module, PureNeurons.
ProofPaper ↗
Key points
- Medical vision-language models (VLMs) have shown increasing potential for clinical image interpretation.
- Existing vision-language alignment methods produce visual representations that fail to preserve anatomical hierarchies and relationships among phenotypic subclasses.
- PureNeurons projects visual representations into the learned latent space, using their positions to selectively aggregate lesion-specific anatomical and phenotypic evidence.
- Experiments on LIDC-IDRI, CBIS-DDSM, and 3DReasonKnee demonstrate that PureVision improves lesion grounding and phenotype characterization in visual question answering and radiology report generation.
Sources (1)
- [1]Geometry-Supervised Visual Representation Learning for Multi-Phenotype Lesion Interpretation in Medical VLMsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 03:22 PM
To address these limitations, we propose PureVision, a geometry-supervised visual representation learning framework for multi-phenotype lesion interpretation in medical VLMs. It combines a geometry-supervised representation learning module, PureEyes, and an anatomy-guided evidence aggregation module, PureNeurons.
Medical vision-language models (VLMs) have shown increasing potential for clinical image interpretation.
Extractive summary: sentences quoted from the sources.