Motahareh Sohrabi

Collaborating researcher - Université de Montréal

Supervisor

Simon Lacoste-Julien

Research Topics

Deep Learning

Optimization

Website

Publications

Accelerated and Stable Convergence with Anchored Generalized Optimistic Method

Jianxin You

We study first-order methods for solving monotone variational inequalities arising in min-max optimization. Classical approaches such as the… (see more) extragradient method rely on two gradient queries per iteration, which limits their analysis and applicability in the online and stochastic settings. We propose a family of Generalized Optimistic Methods with Anchoring (GOMA), which combine two time-scale optimistic updates with an anchoring term inspired by Halpern iteration. In particular, we show that for monotone Lipschitz operators, GOMA achieves an accelerated last-iterate convergence rate of

2025-12-31

International Conference on Machine Learning (Accept (regular))

openreview.net

On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization

Motahareh Sohrabi

Juan Ramirez

Tianyue H. Zhang

Simon Lacoste-Julien

Jose Gallego-Posada

Constrained optimization offers a powerful framework to prescribe desired behaviors in neural network models. Typically, constrained problem… (see more)s are solved via their min-max Lagrangian formulations, which exhibit unstable oscillatory dynamics when optimized using gradient descent-ascent. The adoption of constrained optimization techniques in the machine learning community is currently limited by the lack of reliable, general-purpose update schemes for the Lagrange multipliers. This paper proposes the

2024-07-22

International Conference on Machine Learning (Accept (Poster))

doi.org

proceedings.mlr.press

Weight-Sharing Regularization

Weight-sharing is ubiquitous in deep learning. Motivated by this, we propose a "weight-sharing regularization" penalty on the weights …

2024-04-17

Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (published)