ORACLE: Optimizer-Relative Alignment for Constrained LEarning
Building on this view, we introduce ORACLE, which evaluates the native optimizer's realized step through a joint endpoint linearization of heterogeneous constraint families, constructs the resulting alignment in the optimizer's own geometry, bounds its authority, and commits it only after validation.
ProofPaper ↗
Key points
- Constraint handling methods typically intervene before the optimizer acts, by modifying the objective or the gradient.
- Yet momentum, adaptive scaling, and structured preconditioning can substantially reshape that signal before it becomes a parameter update.
- We formulate optimizer relative constrained learning, where constraint compatibility is assessed on the post optimizer update.
- We evaluate ORACLE across eight Partial Differential Equation benchmarks and four optimizers spanning Euclidean, diagonal adaptive, and structured preconditioned geometries, where it improves or matches native optimizer in 94% of configurations.
Sources (1)
- [1]ORACLE: Optimizer-Relative Alignment for Constrained LEarningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 07:41 PM
Building on this view, we introduce ORACLE, which evaluates the native optimizer's realized step through a joint endpoint linearization of heterogeneous constraint families, constructs the resulting alignment in the optimizer's own geometry, bounds its authority, and commits it only after validation.
Constraint handling methods typically intervene before the optimizer acts, by modifying the objective or the gradient.
Extractive summary: sentences quoted from the sources.