Portrait de Arna Ghosh

Arna Ghosh

ghosharn@mila.quebec
Collaborateur·rice de recherche - McGill
Superviseur⋅e principal⋅e
Blake Richards
Sujets de recherche
Apprentissage de représentations
Apprentissage profond
Neurosciences computationnelles
Systèmes dynamiques
Théorie de l'apprentissage automatique
Vision par ordinateur
Site web
Google Scholar
GitHub

Publications

α-ReQ: Assessing representation quality by measuring eigenspectrum decay
Kumar Krishna Agrawal
Arnab Kumar Mondal
Arna Ghosh
Blake A. Richards
2022-11-28
Conference on Neural Information Processing Systems (Accept)
openreview.net
openreview.net
Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules
Yuhan Helena Liu
Arna Ghosh
Blake A. Richards
Eric Shea-Brown
Guillaume Lajoie
To unveil how the brain learns, ongoing work seeks biologically-plausible approximations of gradient descent algorithms for training recurre… (voir plus)nt neural networks (RNNs). Yet, beyond task accuracy, it is unclear if such learning rules converge to solutions that exhibit different levels of generalization than their nonbiologically-plausible counterparts. Leveraging results from deep learning theory based on loss landscape curvature, we ask: how do biologically-plausible gradient approximations affect generalization? We first demonstrate that state-of-the-art biologically-plausible learning rules for training RNNs exhibit worse and more variable generalization performance compared to their machine learning counterparts that follow the true gradient more closely. Next, we verify that such generalization performance is correlated significantly with loss landscape curvature, and we show that biologically-plausible learning rules tend to approach high-curvature regions in synaptic weight space. Using tools from dynamical systems, we derive theoretical arguments and present a theorem explaining this phenomenon. This predicts our numerical results, and explains why biologically-plausible rules lead to worse and more variable generalization properties. Finally, we suggest potential remedies that could be used by the brain to mitigate this effect. To our knowledge, our analysis is the first to identify the reason for this generalization gap between artificial and biologically-plausible learning rules, which can help guide future investigations into how the brain learns solutions that generalize.
2022-11-28
Conference on Neural Information Processing Systems (Accept)
doi.org
openreview.net
Current State and Future Directions for Learning in Biological Recurrent Neural Networks: A Perspective Piece
Luke Y. Prince
Roy Henha Eyono
Ellen Boven
Arna Ghosh
Joe Pemberton
Franz Scherr
Claudia Clopath
Rui Ponte Costa
Wolfgang Maass
Blake A. Richards
Cristina Savin
Katharina Anna Wilmes
We provide a brief review of the common assumptions about biological learning with findings from experimental neuroscience and contrast them… (voir plus) with the efficiency of gradient-based learning in recurrent neural networks. The key issues discussed in this review include: synaptic plasticity, neural circuits, theory-experiment divide, and objective functions. We conclude with recommendations for both theoretical and experimental neuroscientists when designing new studies that could help bring clarity to these issues.
2022-04-26
Neurons, Behavior, Data Analysis, and Theory (publié)
doi.org
arxiv.org