Portrait of Gül Sena Altıntaş

Gül Sena Altıntaş

Alumni
Website
Google Scholar

Publications

The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions
Gül Sena Altıntaş
Devin Kwok
Colin Raffel
David Rolnick
Neural network training is inherently sensitive to initialization and the randomness induced by stochastic gradient descent. However, it is … (see more)unclear to what extent such effects lead to meaningfully different networks, either in terms of the models’ weights or the underlying functions that were learned. In this work, we show that during the initial "chaotic" phase of training, even extremely small perturbations reliably causes otherwise identical training trajectories to diverge-an effect that diminishes rapidly over training time. We quantify this divergence through (i)
2025-10-06
Proceedings of the 42nd International Conference on Machine Learning (published)
proceedings.mlr.press
The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions
Gül Sena Altıntaş
Devin Kwok
Colin Raffel
David Rolnick
Neural network training is inherently sensitive to initialization and the randomness induced by stochastic gradient descent. However, it is … (see more)unclear to what extent such effects lead to meaningfully different networks, either in terms of the models' weights or the underlying functions that were learned. In this work, we show that during the initial "chaotic" phase of training, even extremely small perturbations reliably causes otherwise identical training trajectories to diverge-an effect that diminishes rapidly over training time. We quantify this divergence through (i)
2025-10-06
Proceedings of the 42nd International Conference on Machine Learning (published)
doi.org
openreview.net
The Butterfly Effect: Tiny Perturbations Cause Neural Network Training to Diverge
Gül Sena Altıntaş
Devin Kwok
David Rolnick
Neural network training begins with a chaotic phase in which the network is sensitive to small perturbations, such as those caused by stocha… (see more)stic gradient descent (SGD). This sensitivity can cause identically initialized networks to diverge both in parameter space and functional similarity. However, the exact degree to which networks are sensitive to perturbation, and the sensitivity of networks as they transition out of the chaotic phase, is unclear. To address this uncertainty, we apply a controlled perturbation at a single point in training time and measure its effect on otherwise identical training trajectories. We find that both the
2024-06-16
ICML.cc/2024/Workshop/HiLD (poster)
openreview.net
openreview.net