Portrait de Hattie Zhou n'est pas disponible

Hattie Zhou

mengfei.zhou@mila.quebec
Doctorat - Université de Montréal
Superviseur⋅e principal⋅e
Hugo Larochelle
Co-superviseur⋅e
Aaron Courville

Publications

Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok
Tikeng Notsawo Pascal Junior
Pascal Notsawo
Hattie Zhou
Mohammad Pezeshki
Irina Rish
Guillaume Dumas
2024-03-04
ICLR.cc/2024/Workshop/ME-FoMo (poster)
doi.org
openreview.net
Teaching Algorithmic Reasoning via In-context Learning
Hattie Zhou
Azade Nova
Aaron Courville
Hugo Larochelle
Behnam Neyshabur
Hanie Sedghi
2022-11-15
ArXiv (preprint)
doi.org
openreview.net
Fortuitous Forgetting in Connectionist Networks
Hattie Zhou
Ankit Vani
Hugo Larochelle
Aaron Courville
Forgetting is often seen as an unwanted characteristic in both human and machine learning. However, we propose that forgetting can in fact b… (voir plus)e favorable to learning. We introduce"forget-and-relearn"as a powerful paradigm for shaping the learning trajectories of artificial neural networks. In this process, the forgetting step selectively removes undesirable information from the model, and the relearning step reinforces features that are consistently useful under different conditions. The forget-and-relearn framework unifies many existing iterative training algorithms in the image classification and language emergence literature, and allows us to understand the success of these algorithms in terms of the disproportionate forgetting of undesirable information. We leverage this understanding to improve upon existing algorithms by designing more targeted forgetting operations. Insights from our analysis provide a coherent view on the dynamics of iterative training in neural networks and offer a clear path towards performance improvements.
2022-01-28
ICLR.cc/2022/Conference (poster)
openreview.net
openreview.net