DSReg: Provably Recovering Individual World Latents without Reconstruction
Methods that recover individual latent variables of the world, from nonlinear ICA to dictionary learning and causal representation learning, anchor the latents to observations through reconstruction, auxiliary supervision, or distributional asymmetries such as non-Gaussianity.
Key points
- Methods without these anchors, including joint-embedding predictive architectures (JEPAs), identify the latent state only up to a linear transformation, so individual latents remain mixed.
- We close this gap: individual world latents can be provably recovered with no reconstruction, no decoder, and no labels.
- Building on the linear identifiability that LeJEPA provides, we prove that under Structural Diversity, DSReg (Dependency-Sparsity Regularization) recovers individual world latents up to signed permutation, without reconstruction or a decoder.
- Across synthetic regimes, world model probes, learned visual encoders, and external renderers, DSReg preserves dense prediction while improving individual-latent recovery and downstream use with scales.
Sources (2)
- [1]DSReg: Provably Recovering Individual World Latents without ReconstructionHugging Face Daily Papers · Oct 7, 12:00 AM
Methods that recover individual latent variables of the world, from nonlinear ICA to dictionary learning and causal representation learning, anchor the latents to observations through reconstruction, auxiliary supervision, or distributional asymmetries such as non-Gaussianity.
Methods without these anchors, including joint-embedding predictive architectures (JEPAs), identify the latent state only up to a linear transformation, so individual latents remain mixed.
- [2]DSReg: Provably Recovering Individual World Latents without ReconstructionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 05:12 AM · same content
Extractive summary: sentences quoted from the sources.