Portrait of Gaurav Iyer is unavailable

Gaurav Iyer

gaurav.iyer@mila.quebec
Master's Research - McGill University
Supervisor
David Rolnick
Google Scholar

Publications

Maximal Initial Learning Rates in Deep ReLU Networks
Gaurav Iyer
Boris Hanin
David Rolnick
2023-07-03
Proceedings of the 40th International Conference on Machine Learning (published)
doi.org
openreview.net
Maximal Initial Learning Rates in Deep ReLU Networks
Gaurav Iyer
Boris Hanin
David Rolnick
Training a neural network requires choosing a suitable learning rate, which involves a trade-off between speed and effectiveness of converge… (see more)nce. While there has been considerable theoretical and empirical analysis of how large the learning rate can be, most prior work focuses only on late-stage training. In this work, we introduce the maximal initial learning rate
2023-04-24
ICML.cc/2023/Conference (poster)
doi.org
openreview.net