Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs
GNN-to-MLP distillation aims to retain the predictive accuracy of a message-passing teacher while deploying a graph-free MLP at inference.
ProofPaper ↗
Key points
- Existing methods mainly transfer node-wise predictions or use confidence-based reweighting, but they do not specify where the student should preserve the teacher's graph-induced geometry.
- We show that this omission leads to two spectral failure modes in the student's representation space.
- Motivated by an energy-weighted teacher-student alignment objective, we propose Graph Geometry-aware MLP (G^2MLP), a training-time distillation framework guided by Ollivier-Ricci curvature.
- Across node-classification benchmarks, G^2MLP consistently improves over graph-free distillation baselines, reduces the teacher-student rank gap in both regimes, and transfers without architectural changes to Graph Transformer teachers and link prediction.
Sources (1)
- [1]Distilling Graph Geometry: Knowledge Gap from GNNs to MLPsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 05:56 PM
GNN-to-MLP distillation aims to retain the predictive accuracy of a message-passing teacher while deploying a graph-free MLP at inference.
Existing methods mainly transfer node-wise predictions or use confidence-based reweighting, but they do not specify where the student should preserve the teacher's graph-induced geometry.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 6, 2026Shape-Bayes: Bayesian Inference of Structured Shapes under Visual Ambiguity
- Oct 6, 2026EmbeddingGemma 2: an open, lightweight multimodal embedding model
- Oct 6, 2026Multimodal Knowledge Distillation for Gastric Adenocarcinoma Classification from Whole-Slide Images
- Oct 6, 2026Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight
- Oct 5, 2026LiquidAI/d1-omni-600M
- Oct 2, 2026Limits of Confidence in Diffusion