Portrait de Pingsheng Li

Pingsheng Li

pingsheng.li@mila.quebec
Représentant du laboratoire
Doctorat
Superviseur⋅e principal⋅e
Blake Richards
Co-supervisor
Guillaume Lajoie
Sujets de recherche
Apprentissage de représentations
Apprentissage multimodal
Modèles de fondation
Neurosciences computationnelles
GitHub

Biographie

Je suis doctorant en première année à l’Université McGill, sous la co-direction du professeur Blake Richards et du professeur Guillaume Lajoie. Mes recherches portent sur le développement de modèles de fondation multimodaux et évolutifs pour des données neuronales complexes, dans le but de concevoir de meilleurs décodeurs cérébraux et des interfaces neuronales intelligentes (plus largement dans le domaine du NeuroAI).

Auparavant, j’ai obtenu mon master dans le programme Neuro-X à l’EPFL, en Suisse. Aujourd’hui à Mila, je vise à faciliter la synergie entre le milieu académique et l’industrie afin de promouvoir un avenir meilleur pour la co-évolution entre l’humain et l’IA.

Publications

Exponentiated gradient learning yields brain-like synaptic distributions
Jonathan H. Cornford
Roman Pogodin
Arna Ghosh
Kaiwen Sheng
Brendan A. Bicknell
Olivier Codol
Pingsheng Li
Beverley A. Clark
Guillaume Lajoie
Blake A. Richards
Artificial neural networks trained with gradient descent (GD) are widely used to model brain learning, but GD allows synapses to switch betw… (voir plus)een excitatory and inhibitory and produces weight distributions unlike those observed experimentally. We show that an alternative rule, exponentiated gradient (EG), avoids these problems because its multiplicative, exponential weight updates mathematically prohibit synapses from changing sign and preferentially scale already-large weights, naturally generating log-normal distributions. This mechanism also concentrates task-relevant computation onto a smaller subset of strong connections, which we show makes EG-trained recurrent neural networks more robust to synaptic pruning and better able to learn from sparsely relevant inputs than GD-trained networks, while matching GD's performance on standard cognitive tasks. These findings tie a normative optimization principle directly to observed features of synaptic organization in the brain.
2026-08-31
Neuron (publié)
doi.org
Learning better with Dale's Law: A Spectral Perspective
Pingsheng Li
Jonathan Cornford
Arna Ghosh
Blake Richards
Most recurrent neural networks (RNNs) do not include a fundamental constraint of real neural circuits: Dale’s Law, which implies that neur… (voir plus)ons must be excitatory (E) or inhibitory (I). Dale’s Law is generally absent from RNNs because simply partitioning a standard network’s units into E and I populations impairs learning. However, here we extend a recent feedforward bio-inspired EI network architecture, named Dale’s ANNs, to recurrent networks, and demonstrate that good performance is possible while respecting Dale’s Law. This begs the question: What makes some forms of EI network learn poorly and others learn well? And, why does the simple approach of incorporating Dale’s Law impair learning? Historically the answer was thought to be the sign constraints on EI network parameters, and this was a motivation behind Dale’s ANNs. However, here we show the spectral properties of the recurrent weight matrix at initialisation are more impactful on network performance than sign constraints. We find that simple EI partitioning results in a singular value distribution that is multimodal and dispersed, whereas standard RNNs have an unimodal, more clustered singular value distribution, as do recurrent Dale’s ANNs. We also show that the spectral properties and performance of partitioned EI networks are worse for small networks with fewer I units, and we present normalised SVD entropy as a measure of spectrum pathology that correlates with performance. Overall, this work sheds light on a long-standing mystery in neuroscience-inspired AI and computational neuroscience, paving the way for greater alignment between neural networks and biology.
2023-12-11
Conference on Neural Information Processing Systems (Accept (poster))
doi.org
openreview.net