Geometry-Aware Multi-Objective Optimization: Norm-Constrained Oracles with Entropic Weight Updates

Key points

• Multiple-gradient descent is typically formulated in Euclidean geometry, whereas matrix-aware optimizers use directions induced by other norms.1
• For any finite-dimensional norm pair, we establish an $O(T^{-1/2})$ ergodic stationarity rate in the deterministic setting, deriving the required trajectory gradient bound from smoothness and lower-boundedness in the unregularized case.1
• With stochastic gradients and a bounded-gradient assumption, one persistent blended-gradient momentum buffer yields an $O(T^{-1/4})$ rate whose leading term has no explicit dependence on the number of tasks.1
• We also establish static regret for the inner weighting objective on arbitrary bounded gradient sequences and, with entropy regularization and a weight floor, iteration-wise inner-gap control after burn-in while retaining ergodic stationarity, conditional on a trajectory gradient bound.1

Sources (1)

• [1]Geometry-Aware Multi-Objective Optimization: Norm-Constrained Oracles with Entropic Weight Updates
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 03:55 PM

  We analyze a single-loop multi-objective method combining a norm-constrained linear minimization oracle with entropic updates of task weights.

  Multiple-gradient descent is typically formulated in Euclidean geometry, whereas matrix-aware optimizers use directions induced by other norms.

Extractive summary: sentences quoted from the sources.