The upcoming meeting, taking place on November 10 at Mila, will explore how we can collectively develop, govern, and deploy high-performing, reliable, and secure agentic systems by connecting academic researchers, industry experts, and practitioners.
Join us on the Venture Scientist Bootcamp, a full time, 4-month incubator at Mila, built specifically for deep tech founders with elite STEM backgrounds. You have until November 21, 2026 to apply!
We use cookies to analyze the browsing and usage of our website and to personalize your experience. You can disable these technologies at any time, but this may limit certain functionalities of the site. Read our Privacy Policy for more information.
Setting cookies
You can enable and disable the types of cookies you wish to accept. However certain choices you make could affect the services offered on our sites (e.g. suggestions, personalised ads, etc.).
Essential cookies
These cookies are necessary for the operation of the site and cannot be deactivated. (Still active)
Analytics cookies
Do you accept the use of cookies to measure the audience of our sites?
Multimedia Player
Do you accept the use of cookies to display and allow you to watch the video content hosted by our partners (YouTube, etc.)?
Leading theories of cortical function posit that sensory processing involves a comparison between top-down predictions and bottom-up sensory… (see more) data. Tests of these theories have been hampered by experimental challenges related to simultaneously recording top-down and bottom-up signals in response to sensory stimuli that either do or do not match internally-generated predictions. Here, we conduct dual-plane 2-photon calcium imaging near-simultaneously from distal apical dendrites and somata of pyramidal neurons in each of 4 different regions of mouse visual cortex while the mice are presented with visual sequences that can either match or mismatch previously habituated visual stimuli. Specifically, we examine the responses in the distal apical dendrites and cell bodies of pyramidal neurons while mice view either movie clips that reverse unexpectedly or Gabor patch sequences that have orientations that do not match habituated patterns. We find that in response to unexpected reversals in natural movies, both the apical dendrites and somata show increased activity that decreases over repeated exposures, consistent with a prediction error response. However, we do not find any clear evidence for increased activity in response to mismatches in Gabor sequences, beyond possible release from adaptation effects. We additionally use volumetric images to connect dendritic and somatic compartments and examine the coupling between connected dendrites and cell bodies in layer 5 pyramidal neurons. We do not find evidence for changes in coupling in response to mismatch stimuli, either for natural movies or Gabor sequences. Our results are consistent across four regions of visual cortex. We also observe that increased responses to reversals in natural movies are stronger in layer 2/3 than layer 5 pyramidal neurons. Altogether, these data suggest that visual cortex is sensitive to deviations from previously-experienced naturalistic inputs. Our data do not suggest that this mismatch sensitivity extends to artificial stimuli, nor do they support theories of prediction error causing changes in the coupling between apical dendrites and somata.
Artificial neural networks trained with gradient descent (GD) are widely used to model brain learning, but GD allows synapses to switch betw… (see more)een excitatory and inhibitory and produces weight distributions unlike those observed experimentally. We show that an alternative rule, exponentiated gradient (EG), avoids these problems because its multiplicative, exponential weight updates mathematically prohibit synapses from changing sign and preferentially scale already-large weights, naturally generating log-normal distributions. This mechanism also concentrates task-relevant computation onto a smaller subset of strong connections, which we show makes EG-trained recurrent neural networks more robust to synaptic pruning and better able to learn from sparsely relevant inputs than GD-trained networks, while matching GD's performance on standard cognitive tasks. These findings tie a normative optimization principle directly to observed features of synaptic organization in the brain.
Computational neuroscience relies on gradient descent (GD) for training artificial neural network (ANN) models of the brain. The advantage o… (see more)f GD is that it is effective at learning difficult tasks. However, it produces ANNs that are a poor phenomenological fit to biology, making them less relevant as models of the brain. Specifically, it violates Dale’s law, by allowing synapses to change from excitatory to inhibitory, and leads to synaptic weights that are not log-normally distributed, contradicting experimental data. Here, starting from first principles of optimisation theory, we present an alternative learning algorithm, exponentiated gradient (EG), that respects Dale’s Law and produces log-normal weights, without losing the power of learning with gradients. We also show that in biologically relevant settings EG outperforms GD, including learning from sparsely relevant signals and dealing with synaptic pruning. Altogether, our results show that EG is a superior learning algorithm for modelling the brain with ANNs.
A growing literature in computational neuroscience leverages gradient descent and learning algorithms that approximate it to study synaptic … (see more)plasticity in the brain. However, the vast majority of this work ignores a critical underlying assumption: the choice of distance for synaptic changes - i.e. the geometry of synaptic plasticity. Gradient descent assumes that the distance is Euclidean, but many other distances are possible, and there is no reason that biology necessarily uses Euclidean geometry. Here, using the theoretical tools provided by mirror descent, we show that the distribution of synaptic weights will depend on the geometry of synaptic plasticity. We use these results to show that experimentally-observed log-normal weight distributions found in several brain areas are not consistent with standard gradient descent (i.e. a Euclidean geometry), but rather with non-Euclidean distances. Finally, we show that it should be possible to experimentally test for different synaptic geometries by comparing synaptic weight distributions before and after learning. Overall, our work shows that the current paradigm in theoretical work on synaptic plasticity that assumes Euclidean synaptic geometry may be misguided and that it should be possible to experimentally determine the true geometry of synaptic plasticity in the brain.
2024-05-06
International Conference on Learning Representations (Accept (spotlight))