Correlational Training of Morphological Neural Networks
In this work, we propose a novel weight update method for morphological neural networks inspired from the Multiplicative Weights Update (MWU) scheme.
ProofPaper ↗
Key points
- Neural networks are typically trained using first-order methods and back-propagation.
- It is unclear whether this approach is optimal for morphological layers whose weight Jacobians are sparse and whose resulting parameter gradients can be poor.
- We view each morphological perceptron as an instance of the learning from experts' advice problem in logarithmic space, and use a correlation-based reward that favors inputs aligned with the desired output change, regardless of whether a strong gradient signal has reached their weight.
- We empirically evaluate our approach by training fully connected layers both as stand-alone models and as parts of larger transformer networks.
Sources (1)
- [1]Correlational Training of Morphological Neural NetworksarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 11:33 AM
In this work, we propose a novel weight update method for morphological neural networks inspired from the Multiplicative Weights Update (MWU) scheme.
Neural networks are typically trained using first-order methods and back-propagation.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 8, 2026LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation
- Oct 8, 2026One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
- Oct 8, 2026Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals
- Oct 7, 2026Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning
- Oct 6, 2026huggingface/transformers v5.19.0: Release v5.19.0
- Oct 6, 2026EmbeddingGemma 2: an open, lightweight multimodal embedding model