Gaurav Iyer

Research Topics

Deep Learning

Google Scholar

Publications

Linear Weight Interpolation Leads to Transient Performance Gains

Gintare Karolina Dziugaite

2024-06-16

ICML.cc/2024/Workshop/HiLD (poster)

Linear Weight Interpolation Leads to Transient Performance Gains

Gintare Karolina Dziugaite

2024-01-01

Trans. Mach. Learn. Res. (published)

Maximal Initial Learning Rates in Deep ReLU Networks

Boris Hanin

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (published)

doi.org

Maximal Initial Learning Rates in Deep ReLU Networks

Boris Hanin

Training a neural network requires choosing a suitable learning rate, which involves a trade-off between speed and effectiveness of converge… (see more)nce. While there has been considerable theoretical and empirical analysis of how large the learning rate can be, most prior work focuses only on late-stage training. In this work, we introduce the maximal initial learning rate

2023-04-24

ICML.cc/2023/Conference (poster)

doi.org