Publications

Systematic detection of brain protein-coding genes under positive selection during primate evolution and their roles in cognition
Guillaume Dumas
Simon Malesys
Thomas Bourgeron
The human brain differs from that of other primates, but the genetic basis of these differences remains unclear. We investigated the evoluti… (see more)onary pressures acting on almost all human protein-coding genes (N = 11,667; 1:1 orthologs in primates) based on their divergence from those of early hominins, such as Neanderthals, and non-human primates. We confirm that genes encoding brain-related proteins are among the most strongly conserved protein-coding genes in the human genome. Combining our evolutionary pressure metrics for the protein-coding genome with recent data sets, we found that this conservation applied to genes functionally associated with the synapse and expressed in brain structures such as the prefrontal cortex and the cerebellum. Conversely, several genes presenting signatures commonly associated with positive selection appear as causing brain diseases or conditions, such as micro/macrocephaly, Joubert syndrome, dyslexia, and autism. Among those, a number of DNA damage response genes associated with microcephaly in humans such as BRCA1, NHEJ1, TOP3A, and RNF168 show strong signs of positive selection and might have played a role in human brain size expansion during primate evolution. We also showed that cerebellum granule neurons express a set of genes also presenting signatures of positive selection and that may have contributed to the emergence of fine motor skills and social cognition in humans. This resource is available online and can be used to estimate evolutionary constraints acting on a set of genes and to explore their relative contributions to human traits.
2021-01-12
Genome Research (published)
doi.org
Adversarial score matching and improved sampling for image generation
Alexia Jolicoeur-Martineau
Rémi Piché-Taillefer
Remi Tachet des Combes
Ioannis Mitliagkas
Denoising Score Matching with Annealed Langevin Sampling (DSM-ALS) has recently found success in generative modeling. The approach works by … (see more)first training a neural network to estimate the score of a distribution, and then using Langevin dynamics to sample from the data distribution assumed by the score network. Despite the convincing visual quality of samples, this method appears to perform worse than Generative Adversarial Networks (GANs) under the Fréchet Inception Distance, a standard metric for generative models. We show that this apparent gap vanishes when denoising the final Langevin samples using the score network. In addition, we propose two improvements to DSM-ALS: 1) Consistent Annealed Sampling as a more stable alternative to Annealed Langevin Sampling, and 2) a hybrid training formulation, composed of both Denoising Score Matching and adversarial objectives. By combining these two techniques and exploring different network architectures, we elevate score matching methods and obtain results competitive with state-of-the-art image generation on CIFAR-10.
2021-01-11
ICLR.cc/2021/Conference (poster)
openreview.net
openreview.net
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
Ossama Ahmed
Frederik Träuble
Anirudh Goyal
Alexander Neitz
Manuel Wuthrich
Yoshua Bengio
Bernhard Schölkopf
Stefan Bauer
Despite recent successes of reinforcement learning (RL), it remains a challenge for agents to transfer learned skills to related environment… (see more)s. To facilitate research addressing this problem, we propose CausalWorld, a benchmark for causal structure and transfer learning in a robotic manipulation environment. The environment is a simulation of an open-source robotic platform, hence offering the possibility of sim-to-real transfer. Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures. The key strength of CausalWorld is that it provides a combinatorial family of such tasks with common causal structure and underlying factors (including, e.g., robot and object masses, colors, sizes). The user (or the agent) may intervene on all causal variables, which allows for fine-grained control over how similar different tasks (or task distributions) are. One can thus easily define training and evaluation distributions of a desired difficulty level, targeting a specific form of generalization (e.g., only changes in appearance or object mass). Further, this common parametrization facilitates defining curricula by interpolating between an initial and a target task. While users may define their own task distributions, we present eight meaningful distributions as concrete benchmarks, ranging from simple to very challenging, all of which require long-horizon planning as well as precise low-level motor control. Finally, we provide baseline results for a subset of these tasks on distinct training curricula and corresponding evaluation protocols, verifying the feasibility of the tasks in this benchmark.
2021-01-11
ICLR.cc/2021/Conference (poster)
doi.org
openreview.net
Integrating Categorical Semantics into Unsupervised Domain Translation
Samuel Lavoie
Faruk Ahmed
Aaron Courville
While unsupervised domain translation (UDT) has seen a lot of success recently, we argue that mediating its translation via categorical sema… (see more)ntic features could broaden its applicability. In particular, we demonstrate that categorical semantics improves the translation between perceptually different domains sharing multiple object categories. We propose a method to learn, in an unsupervised manner, categorical semantic features (such as object labels) that are invariant of the source and target domains. We show that conditioning the style encoder of unsupervised domain translation methods on the learned categorical semantics leads to a translation preserving the digits on MNIST
2021-01-11
ICLR.cc/2021/Conference (poster)
doi.org
openreview.net
Iterated Learning for Emergent Systematicity in VQA
Ankit Vani
Max Schwarzer
Yuchen Lu
Eeshan Dhekane
Aaron Courville
Although neural module networks have an architectural bias towards compositionality, they require gold standard layouts to generalize system… (see more)atically in practice. When instead learning layouts and modules jointly, compositionality does not arise automatically and an explicit pressure is necessary for the emergence of layouts exhibiting the right structure. We propose to address this problem using iterated learning, a cognitive science theory of the emergence of compositional languages in nature that has primarily been applied to simple referential games in machine learning. Considering the layouts of module networks as samples from an emergent language, we use iterated learning to encourage the development of structure within this language. We show that the resulting layouts support systematic generalization in neural agents solving the more complex task of visual question-answering. Our regularized iterated learning method can outperform baselines without iterated learning on SHAPES-SyGeT (SHAPES Systematic Generalization Test), a new split of the SHAPES dataset we introduce to evaluate systematic generalization, and on CLOSURE, an extension of CLEVR also designed to test systematic generalization. We demonstrate superior performance in recovering ground-truth compositional program structure with limited supervision on both SHAPES-SyGeT and CLEVR.
2021-01-11
ICLR.cc/2021/Conference (oral)
doi.org
openreview.net
Learning to Live with Dale's Principle: ANNs with Separate Excitatory and Inhibitory Units
Jonathan Cornford
Damjan Kalajdzievski
Marco Leite
Amélie Lamarquette
Dimitri M. Kullmann
Blake Richards
A bstract The units in artificial neural networks (ANNs) can be thought of as abstraction… (see more)s of biological neurons, and ANNs are increasingly used in neuroscience research. However, there are many important differences between ANN units and real neurons. One of the most notable is the absence of Dale’s principle, which ensures that biological neurons are either exclusively excitatory or inhibitory. Dale’s principle is typically left out of ANNs because its inclusion impairs learning. This is problematic, because one of the great advantages of ANNs for neuroscience research is their ability to learn complicated, realistic tasks. Here, by taking inspiration from feedforward inhibitory interneurons in the brain we show that we can develop ANNs with separate populations of excitatory and inhibitory units that learn just as well as standard ANNs. We call these networks Dale’s ANNs (DANNs). We present two insights that enable DANNs to learn well: (1) DANNs are related to normalization schemes, and can be initialized such that the inhibition centres and standardizes the excitatory activity, (2) updates to inhibitory neuron parameters should be scaled using corrections based on the Fisher Information matrix. These results demonstrate how ANNs that respect Dale’s principle can be built without sacrificing learning performance, which is important for future work using ANNs as models of the brain. The results may also have interesting implications for how inhibitory plasticity in the real brain operates.
2021-01-11
ICLR.cc/2021/Conference (poster)
doi.org
openreview.net
Recurrent Independent Mechanisms
Anirudh Goyal
Alex Lamb
Jordan Hoffmann
Shagun Sodhani
Sergey Levine
Yoshua Bengio
Bernhard Schölkopf
Learning modular structures which reflect the dynamics of the environment can lead to better generalization and robustness to changes which … (see more)only affect a few of the underlying causes. We propose Recurrent Independent Mechanisms (RIMs), a new recurrent architecture in which multiple groups of recurrent cells operate with nearly independent transition dynamics, communicate only sparingly through the bottleneck of attention, and are only updated at time steps where they are most relevant. We show that this leads to specialization amongst the RIMs, which in turn allows for dramatically improved generalization on tasks where some factors of variation differ systematically between training and evaluation.
2021-01-11
ICLR.cc/2021/Conference (spotlight)
doi.org
openreview.net
RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs
Meng Qu
Junkun Chen
Louis-Pascal Xhonneux
Yoshua Bengio
Jian Tang
This paper studies learning logic rules for reasoning on knowledge graphs. Logic rules provide interpretable explanations when used for pred… (see more)iction as well as being able to generalize to other tasks, and hence are critical to learn. Existing methods either suffer from the problem of searching in a large search space (e.g., neural logic programming) or ineffective optimization due to sparse rewards (e.g., techniques based on reinforcement learning). To address these limitations, this paper proposes a probabilistic model called RNNLogic. RNNLogic treats logic rules as a latent variable, and simultaneously trains a rule generator as well as a reasoning predictor with logic rules. We develop an EM-based algorithm for optimization. In each iteration, the reasoning predictor is first updated to explore some generated logic rules for reasoning. Then in the E-step, we select a set of high-quality rules from all generated rules with both the rule generator and reasoning predictor via posterior inference; and in the M-step, the rule generator is updated with the rules selected in the E-step. Experiments on four datasets prove the effectiveness of RNNLogic.
2021-01-11
ICLR.cc/2021/Conference (poster)
doi.org
openreview.net
Spatially Structured Recurrent Modules
Nasim Rahaman
Anirudh Goyal
Muhammad Waleed Gondal
Manuel Wuthrich
Stefan Bauer
Yash Sharma
Yoshua Bengio
Bernhard Schölkopf
Capturing the structure of a data-generating process by means of appropriate inductive biases can help in learning models that generalise we… (see more)ll and are robust to changes in the input distribution. While methods that harness spatial and temporal structures find broad application, recent work has demonstrated the potential of models that leverage sparse and modular structure using an ensemble of sparingly interacting modules. In this work, we take a step towards dynamic models that are capable of simultaneously exploiting both modular and spatiotemporal structures. To this end, we model the dynamical system as a collection of autonomous but sparsely interacting sub-systems that interact according to a learned topology which is informed by the spatial structure of the underlying system. This gives rise to a class of models that are well suited for capturing the dynamics of systems that only offer local views into their state, along with corresponding spatial locations of those views. On the tasks of video prediction from cropped frames and multi-agent world modelling from partial observations in the challenging Starcraft2 domain, we find our models to be more robust to the number of available views and better capable of generalisation to novel tasks without additional training than strong baselines that perform equally well or better on the training distribution.
2021-01-11
ICLR.cc/2021/Conference (poster)
openreview.net
openreview.net
Mass‐spectrometry analysis of the human pineal proteome during night and day and in autism
Guillaume Dumas
Hany Goubran‐Botros
Mariette Matondo
Cécile Pagan
Cyril Boulègue
Thibault Chaze
Julia Chamot‐Rooke
Erik Maronde
Thomas Bourgeron
The human pineal gland regulates day‐night dynamics of multiple physiological processes, especially through the secretion of melatonin. Us… (see more)ing mass‐spectrometry‐based proteomics and dedicated analysis tools, we identify proteins in the human pineal gland and analyze systematically their variation throughout the day and compare these changes in the pineal proteome between control specimens and donors diagnosed with autism. Results reveal diverse regulated clusters of proteins with, among others, catabolic carbohydrate process and cytoplasmic membrane‐bounded vesicle‐related proteins differing between day and night and/or control versus autism pineal glands. These data show novel and unexpected processes happening in the human pineal gland during the day/night rhythm as well as specific differences between autism donor pineal glands and those from controls.
2021-01-10
Journal of Pineal Research (published)
doi.org
Attention Based Pruning for Shift Networks
Ghouthi Boukli hacene
Carlos Lassance
Vincent Gripon
Matthieu Courbariaux
Yoshua Bengio
In many application domains such as computer vision, Convolutional Layers (CLs) are key to the accuracy of deep learning methods. However, i… (see more)t is often required to assemble a large number of CLs, each containing thousands of parameters, in order to reach state-of-the-art accuracy, thus resulting in complex and demanding systems that are poorly fitted to resource-limited devices. Recently, methods have been proposed to replace the generic convolution operator by the combination of a shift operation and a simpler 1x1 convolution. The resulting block, called Shift Layer (SL), is an efficient alternative to CLs in the sense it allows to reach similar accuracies on various tasks with faster computations and fewer parameters. In this contribution, we introduce Shift Attention Layers (SALs), which extend SLs by using an attention mechanism that learns which shifts are the best at the same time the network function is trained. We demonstrate SALs are able to outperform vanilla SLs (and CLs) on various object recognition benchmarks while significantly reducing the number of float operations and parameters for the inference.
2021-01-09
International Conference on Pattern Recognition (published)
doi.org
arxiv.org
The patient advisor, an organizational resource as a lever for an enhanced oncology patient experience (PAROLE-onco): a longitudinal multiple case study protocol
M. P. Pomey
M. de Guise
M. Desforges
K. Bouchard
C. Vialaron
L. Normandin
M. Iliescu-Nelea
I. Fortin
I. Ganache
C. Régis
Z. Rosberger
D. Charpentier
L. Bélanger
M. Dorval
D. P. Ghadiri
M. Lavoie-Tremblay
A. Boivin
J. F. Pelletier
N. Fernandez
A. M. Danino
Quebec is one of the Canadian provinces with the highest rates of cancer incidence and prevalence. A study by the Rossy Cancer Network (RCN)… (see more) of McGill university assessed six aspects of the patient experience among cancer patients and found that emotional support is the aspect most lacking. To improve this support, trained patient advisors (PAs) can be included as full-fledged members of the healthcare team, given that PA can rely on their knowledge with experiencing the disease and from using health and social care services to accompany cancer patients, they could help to round out the health and social care services offer in oncology. However, the feasibility of integrating PAs in clinical oncology teams has not been studied. In this multisite study, we will explore how to integrate PAs in clinical oncology teams and, under what conditions this can be successfully done. We aim to better understand effects of this PA intervention on patients, on the PAs themselves, the health and social care team, the administrators, and on the organization of services and to identify associated ethical and legal issues. We will conduct six mixed methods longitudinal case studies. Qualitative data will be used to study the integration of the PAs into clinical oncology teams and to identify the factors that are facilitators and inhibitors of the process, the associated ethical and legal issues, and the challenges that the PAs experience. Quantitative data will be used to assess effects on patients, PAs and team members, if any, of the PA intervention. The results will be used to support oncology programs in the integration of PAs into their healthcare teams and to design a future randomized pragmatic trial to evaluate the impact of PAs as full-fledged members of clinical oncology teams on cancer patients’ experience of emotional support throughout their care trajectory. This study will be the first to integrate PAs as full-fledged members of the clinical oncology team and to assess possible clinical and organizational level effects. Given the unique role of PAs, this study will complement the body of research on peer support and patient navigation. An additional innovative aspect of this study will be consideration of the ethical and legal issues at stake and how to address them in the health care organizations.
2021-01-03
BMC Health Services Research (published)
doi.org