Stochastic Mirror Descent
No mentions found
This entity hasn't been tracked yet, or Iris is still building its knowledge base.
Related Articles from SNS
Bregman meets L\'evy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time
arXiv:2606.03769v1 Announce Type: cross Abstract: We study the robustness of stochastic mirror descent (SMD) under heavy-tailed noise, focusing on whether the method retains its convergence guarantees when run with infinite-variance stochastic gradient input. To address this question in a principled manner, we begin by introducing a continuous-time model of SMD as a stochastic differential equation (SDE) driven by a centered L\'evy noise process with finite $p$-th order moments, $1 < p \leq...
A Unified Variational Design of Predictive Mirror Descent in Convex Games under Stochastic Feedback
Announce Type: cross Abstract: Mirror descent provides a geometric framework for learning in games, but its last-iterate behavior can fail in weakly stable regimes, where the dynamics may exhibit rotational or recurrent transients. Predictive mirror methods mitigate this issue by modifying the feedback entering the mirror update, yet standard predictive variants are typically introduced algorithmically and analyzed one at a time. This letter gives a variational route to predictive feedback...
Mirror Descent Under Generalized Smoothness
arXiv:2502.00753v4 Announce Type: replace-cross Abstract: Smoothness is crucial for attaining fast rates in first-order optimization. However, many optimization problems in modern machine learning involve non-smooth objectives. Recent studies relax the smoothness assumption by allowing the Lipschitz constant of the gradient to grow with respect to the gradient norm, which accommodates a broad range of objectives in practice.
In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise
Announce Type: cross Abstract: Many stochastic gradient methods are believed not to converge when the noise in stochastic gradients has only a finite $p$-th moment for $p\in\left(1,2\right)$, a setting known as the heavy-tailed noise assumption. However, some recent studies have found that Stochastic Gradient Descent ($\textsf{SGD}$), without any modification to its update rule, can surprisingly converge in expectation for convex problems with bounded domains, highlighting the potential of...