Residual spectral instabilities in representation learning
Learned representations can lose latent degrees of freedom successively, suggesting a cascade of transitions whose underlying stability principle remains unclear.
ProofPaper ↗
Key points
- Here we formulate dimension-wise posterior collapse in variational autoencoder (VAE) as a fluctuation theory around partially collapsed states.
- We show that the collapsed directions form an invariant fluctuation sector and derive its exact mass spectrum in terms of a conditional residual operator.
- Numerical continuation experiments show successive loss of latent dimensions near these spectral marginalities.
- These results support a spectral cascade interpretation governed by residual information left unexplained by the surviving representation.
Sources (1)
- [1]Residual spectral instabilities in representation learningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:09 AM
Learned representations can lose latent degrees of freedom successively, suggesting a cascade of transitions whose underlying stability principle remains unclear.
Here we formulate dimension-wise posterior collapse in variational autoencoder (VAE) as a fluctuation theory around partially collapsed states.
Extractive summary: sentences quoted from the sources.