Live Roundups Editor's Desk Insights Executive Ops Weather About 🔍
Home › Science › Modeling AdaGrad, RMSProp, and Adam with...
Science

Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations

arXiv CS Monday 08 June 2026, 04:00 UTC By Carlos Heredia 1 min read
Key Points

Announce Type: replace Abstract: In this paper, we propose a continuous-time formulation for the AdaGrad, RMSProp, and Adam optimization algorithms by modeling them as first-order integro-differential equations. We perform numerical simulations of these equations, along with stability and convergence analyses, to demonstrate their validity as accurate approximations of the original algorithms. Our results indicate a strong agreement between the behavior of the continuous-time models and the...

arXiv:2411.09734v3 Announce Type: replace Abstract: In this paper, we propose a continuous-time formulation for the AdaGrad, RMSProp, and Adam optimization algorithms by modeling them as first-order integro-differential equations. We perform numerical simulations of these equations, along with stability and convergence analyses, to demonstrate their validity as accurate approximations of the original algorithms. Our results indicate a strong agreement between the behavior of the continuous-time models and the discrete implementations, thus providing a new perspective on the theoretical understanding of adaptive optimization methods.
Modeling AdaGrad (PERSON) RMSProp (ORG) Adam (PERSON) Integro-Differential (ORG) AdaGrad (PERSON)
Originally published by arXiv CS Read original →

Related Stories

Artificial turf contains 400 chemicals tied to cancer and hormone disruption. But is it unsafe?

Live Science 17m ago

Japan’s Retail Investor Army Flocks to SpaceX After IPO Drought

Bloomberg Technology 36m ago

NASA addresses criticism over all-male crew selected for Artemis III test mission

When NASA’s Artemis III mission launches next year, the crew won’t include any women — a revelation that sparked controversy after the agency on Tuesday announced the four astronauts selected for the flight. “Not a single woman flying on Artemis III is an insane choice,” Alexandra Doten, a space influencer who goes by Astro Alexandra, posted on X on Tuesday. NASA Administrator Jared Isaacman attempted to address these criticisms head-on Wednesday.

NBC News 39m ago

Jeffery Lee breathes ‘sigh of relief’ after Alabama’s nitrogen execution deemed unconstitutional

A death row prisoner whose planned execution on Thursday was suddenly halted became emotional when he learned that a federal court had ruled Alabama’s use of nitrogen gas violates the constitutional ban on cruel and unusual punishment. “It’s like an expected sigh of relief in one aspect, and then you still got to stay and maintain your focus and continue to fight,” Jeffery Lee, who has been on death row for nearly three decades, told NBC News by phone Tuesday. He spoke from the William C....

NBC News 55m ago