Portrait de Baye Malick Gning n'est pas disponible

Baye Malick Gning

malick.gning@mila.quebec
Doctorat - Concordia
Superviseur⋅e principal⋅e
Elvis Dohmatob
Sujets de recherche
Interprétabilité
Modèles probabilistes
Optimisation
Théorie de l'apprentissage automatique
Site web
LinkedIn
GitHub

Publications

Statistical Vignettes on the Grokking Phenomenon
Baye Malick Gning
Elvis Dohmatob
In this work, we study grokking in a simple, analytically tractable setting whose rich phenomenology allows us to disentangle the effects of… (voir plus) initialization, signal strength, weight decay, sample size, and input dimension. These factors can produce superficially similar learning curves through distinct mechanisms, complicating their interpretation. Our analysis distinguishes early fitting through nuisance features, core feature learning, and delayed generalization. We identify sharp conditions under which grokking provably occurs and derive analytic asymptotic expressions for the lengths of the generalization plateaus. Our results also characterize regimes of early generalization and failure to reach a prescribed test accuracy target. We show that the interpolation edge constitutes a distinct dynamical regime, where vanishing spectral gaps separate core learning from generalization, helping clarify the role of proximity to interpolation in existing accounts of grokking. The resulting picture is analogous to fast and slow phases in chemical kinetics and suggests that the lazy-to-rich transition discussed in the literature may reflect a similar separation of dynamical time scales. Our theoretical results are faithfully supported by extensive experiments.
2026-10-12
arXiv (prépublication)
doi.org
arxiv.org