Portrait de Roman Pogodin n'est pas disponible

Roman Pogodin

Alumni
Site web
Google Scholar

Publications

Learning From the Past with Cascading Eligibility Traces
Tokiniaina Raharison Ralambomihanta
Ivan Anokhin
Roman Pogodin
Samira Ebrahimi Kahou
Jonathan Cornford
Blake A. Richards
2026-12-30
OpenReview (publié)
doi.org
openreview.net
Exponentiated gradient learning yields brain-like synaptic distributions
Jonathan H. Cornford
Roman Pogodin
Arna Ghosh
Kaiwen Sheng
Brendan A. Bicknell
Olivier Codol
Pingsheng Li
Beverley A. Clark
Guillaume Lajoie
Blake A. Richards
Artificial neural networks trained with gradient descent (GD) are widely used to model brain learning, but GD allows synapses to switch betw… (voir plus)een excitatory and inhibitory and produces weight distributions unlike those observed experimentally. We show that an alternative rule, exponentiated gradient (EG), avoids these problems because its multiplicative, exponential weight updates mathematically prohibit synapses from changing sign and preferentially scale already-large weights, naturally generating log-normal distributions. This mechanism also concentrates task-relevant computation onto a smaller subset of strong connections, which we show makes EG-trained recurrent neural networks more robust to synaptic pruning and better able to learn from sparsely relevant inputs than GD-trained networks, while matching GD's performance on standard cognitive tasks. These findings tie a normative optimization principle directly to observed features of synaptic organization in the brain.
2026-08-31
Neuron (publié)
doi.org
Brain-like learning with exponentiated gradients
Jonathan Cornford
Roman Pogodin
Arna Ghosh
Kaiwen Sheng
Brendan A. Bicknell
Olivier Codol
Beverley A. Clark
Guillaume Lajoie
Blake A. Richards
Computational neuroscience relies on gradient descent (GD) for training artificial neural network (ANN) models of the brain. The advantage o… (voir plus)f GD is that it is effective at learning difficult tasks. However, it produces ANNs that are a poor phenomenological fit to biology, making them less relevant as models of the brain. Specifically, it violates Dale’s law, by allowing synapses to change from excitatory to inhibitory, and leads to synaptic weights that are not log-normally distributed, contradicting experimental data. Here, starting from first principles of optimisation theory, we present an alternative learning algorithm, exponentiated gradient (EG), that respects Dale’s Law and produces log-normal weights, without losing the power of learning with gradients. We also show that in biologically relevant settings EG outperforms GD, including learning from sparsely relevant signals and dealing with synaptic pruning. Altogether, our results show that EG is a superior learning algorithm for modelling the brain with ANNs.
2024-10-25
bioRxiv (prépublication)
doi.org
Synaptic Weight Distributions Depend on the Geometry of Plasticity
Roman Pogodin
Jonathan Cornford
Arna Ghosh
Gauthier Gidel
Guillaume Lajoie
Blake Aaron Richards
A growing literature in computational neuroscience leverages gradient descent and learning algorithms that approximate it to study synaptic … (voir plus)plasticity in the brain. However, the vast majority of this work ignores a critical underlying assumption: the choice of distance for synaptic changes - i.e. the geometry of synaptic plasticity. Gradient descent assumes that the distance is Euclidean, but many other distances are possible, and there is no reason that biology necessarily uses Euclidean geometry. Here, using the theoretical tools provided by mirror descent, we show that the distribution of synaptic weights will depend on the geometry of synaptic plasticity. We use these results to show that experimentally-observed log-normal weight distributions found in several brain areas are not consistent with standard gradient descent (i.e. a Euclidean geometry), but rather with non-Euclidean distances. Finally, we show that it should be possible to experimentally test for different synaptic geometries by comparing synaptic weight distributions before and after learning. Overall, our work shows that the current paradigm in theoretical work on synaptic plasticity that assumes Euclidean synaptic geometry may be misguided and that it should be possible to experimentally determine the true geometry of synaptic plasticity in the brain.
2024-05-06
International Conference on Learning Representations (Accept (spotlight))
doi.org
openreview.net