Portrait de Gaurav Iyer n'est pas disponible

Gaurav Iyer

gaurav.iyer@mila.quebec
Maîtrise recherche - McGill University
Superviseur⋅e principal⋅e
David Rolnick
Google Scholar

Publications

Maximal Initial Learning Rates in Deep ReLU Networks
Gaurav Iyer
Boris Hanin
David Rolnick
2023-07-03
Proceedings of the 40th International Conference on Machine Learning (publié)
doi.org
openreview.net
Maximal Initial Learning Rates in Deep ReLU Networks
Gaurav Iyer
Boris Hanin
David Rolnick
Training a neural network requires choosing a suitable learning rate, which involves a trade-off between speed and effectiveness of converge… (voir plus)nce. While there has been considerable theoretical and empirical analysis of how large the learning rate can be, most prior work focuses only on late-stage training. In this work, we introduce the maximal initial learning rate
2023-04-24
ICML.cc/2023/Conference (poster)
doi.org
openreview.net