Representation Alignment from a Few Paired Data via Distilled Optimal Transport
Aligning two independently pretrained unimodal encoders typically demands millions of paired examples.
ProofPaper ↗
Key points
- In this paper, we study the regime where only $N \sim 100$--$500$ pairs are available, while unpaired data on either side is abundant.
- We additionally put this marginal constraint in relation with the one-step hub corrections of the bilingual lexicon induction literature.
- Pairing is required only for the anchors: the unpaired collections may be drawn independently from the two marginals, so that no item outside the anchor set needs to have both of its modalities observed.
- We evaluate on image-text and molecule-text alignment, where our method outperforms existing approaches at the same anchor budget on Flickr30k, MS-COCO and ChEBI-20.
Sources (1)
- [1]Representation Alignment from a Few Paired Data via Distilled Optimal TransportarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 10:38 PM
Aligning two independently pretrained unimodal encoders typically demands millions of paired examples.
In this paper, we study the regime where only $N \sim 100$--$500$ pairs are available, while unpaired data on either side is abundant.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 8, 2026huggingface/trl v1.15.0
- Oct 8, 2026When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
- Oct 8, 2026One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
- Oct 8, 2026SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces
- Oct 8, 2026Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models
- Oct 8, 2026ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills