Portrait de Noah Marshall n'est pas disponible

Noah Marshall

noah.marshall@mila.quebec
Doctorat - McGill
Superviseur⋅e principal⋅e
Adam M. Oberman
Co-supervisor
Courtney Paquette
Sujets de recherche
Apprentissage profond

Publications

Phases of Muon: When Muon Eclipses SignSGD
Elliot Paquette
Noah Marshall
Lucas Benigni
Guangyuan Wang
Atish Agarwala
Courtney Paquette
Recently, Muon and related spectral optimizers have demonstrated strong empirical performance as scalable stochastic methods, often outperfo… (voir plus)rming Adam. Yet their behaviour remains poorly understood. We analyze stochastic spectral optimizers, including Muon, on a high-dimensional matrix-valued least squares problem. We derive explicit deterministic dynamics that provide a tractable framework for studying learning behaviour with a focus on (stochastic) SignSVD, which Muon approximates, and (stochastic) SignSGD, the latter serving as a proxy for Adam. Our analysis shows that for large batch size, SignSVD performs a square-root preconditioning with respect to the data covariance spectrum, while for small batch size smaller eigenmodes behave like SGD, slowing down convergence. We contrast with SignSGD which for generic covariance performs no preconditioning and has no transition, leading to different optimal learning rates and convergence characteristics. The two methods match up to a constant factor with isotropic data, but behave differently with anisotropic data. An analysis of a power law covariance model with data exponent
2026-05-09
arXiv (prépublication)
doi.org
arxiv.org
Exact risk curves of signSGD in High-Dimensions: quantifying preconditioning and noise-compression effects
Ke Liang Xiao
Noah Marshall
Atish Agarwala
Elliot Paquette
In recent years, signSGD has garnered interest as both a practical optimizer as well as a simple model to understand adaptive optimizers lik… (voir plus)e Adam. Though there is a general consensus that signSGD acts to precondition optimization and reshapes noise, quantitatively understanding these effects in theoretically solvable settings remains difficult. We present an analysis of signSGD in a high dimensional limit, and derive a limiting SDE and ODE to describe the risk. Using this framework we quantify four effects of signSGD: effective learning rate, noise compression, diagonal preconditioning, and gradient noise reshaping. Our analysis is consistent with experimental observations but moves beyond that by quantifying the dependence of these effects on the data and noise distributions. We conclude with a conjecture on how these results might be extended to Adam.
2025-07-14
International Conference on Machine Learning (Accept (poster))
doi.org
proceedings.mlr.press
FairCal: Fairness Calibration for Face Verification
Tiago Salvador
Stephanie Cairns
Vikram Voleti
Noah Marshall
Adam Oberman
Despite being widely used, face recognition models suffer from bias: the probability of a false positive (incorrect face match) strongly dep… (voir plus)ends on sensitive attributes such as the ethnicity of the face. As a result, these models can disproportionately and negatively impact minority groups, particularly when used by law enforcement. The majority of bias reduction methods have several drawbacks: they use an end-to-end retraining approach, may not be feasible due to privacy issues, and often reduce accuracy. An alternative approach is post-processing methods that build fairer decision classifiers using the features of pre-trained models, thus avoiding the cost of retraining. However, they still have drawbacks: they reduce accuracy (AGENDA, PASS, FTC), or require retuning for different false positive rates (FSN). In this work, we introduce the Fairness Calibration (FairCal) method, a post-training approach that simultaneously: (i) increases model accuracy (improving the state-of-the-art), (ii) produces fairly-calibrated probabilities, (iii) significantly reduces the gap in the false positive rates, (iv) does not require knowledge of the sensitive attribute, and (v) does not require retraining, training an additional model, or retuning. We apply it to the task of Face Verification, and obtain state-of-the-art results with all the above advantages.
2022-01-27
International Conference on Learning Representations (poster)
doi.org
openreview.net