Portrait of Baye Malick Gning is unavailable

Baye Malick Gning

malick.gning@mila.quebec
PhD - Concordia University
Supervisor
Elvis Dohmatob
Research Topics
Interpretability
Machine Learning Theory
Optimization
Probabilistic Models
Website
LinkedIn
GitHub

Publications

Statistical Vignettes on the Grokking Phenomenon
Baye Malick Gning
Elvis Dohmatob
In this work, we study grokking in a simple, analytically tractable setting whose rich phenomenology allows us to disentangle the effects of… (see more) initialization, signal strength, weight decay, sample size, and input dimension. These factors can produce superficially similar learning curves through distinct mechanisms, complicating their interpretation. Our analysis distinguishes early fitting through nuisance features, core feature learning, and delayed generalization. We identify sharp conditions under which grokking provably occurs and derive analytic asymptotic expressions for the lengths of the generalization plateaus. Our results also characterize regimes of early generalization and failure to reach a prescribed test accuracy target. We show that the interpolation edge constitutes a distinct dynamical regime, where vanishing spectral gaps separate core learning from generalization, helping clarify the role of proximity to interpolation in existing accounts of grokking. The resulting picture is analogous to fast and slow phases in chemical kinetics and suggests that the lazy-to-rich transition discussed in the literature may reflect a similar separation of dynamical time scales. Our theoretical results are faithfully supported by extensive experiments.
2026-10-12
arXiv (preprint)
doi.org
arxiv.org