Publications

Parallel inference of hierarchical latent dynamics in two-photon calcium imaging of neuronal populations
Luke Y. Prince
Shahab Bakhtiari
Colleen J. Gillon
Blake A. Richards
Dynamic latent variable modelling has provided a powerful tool for understanding how populations of neurons compute. For spiking data, such … (see more)latent variable modelling can treat the data as a set of point-processes, due to the fact that spiking dynamics occur on a much faster timescale than the computational dynamics being inferred. In contrast, for other experimental techniques, the slow dynamics governing the observed data are similar in timescale to the computational dynamics that researchers want to infer. An example of this is in calcium imaging data, where calcium dynamics can have timescales on the order of hundreds of milliseconds. As such, the successful application of dynamic latent variable modelling to modalities like calcium imaging data will rest on the ability to disentangle the deeper- and shallower-level dynamical systems’ contributions to the data. To-date, no techniques have been developed to directly achieve this. Here we solve this problem by extending recent advances using sequential variational autoencoders for dynamic latent variable modelling of neural data. Our system VaLPACa (Variational Ladders for Parallel Autoencoding of Calcium imaging data) solves the problem of disentangling deeper- and shallower-level dynamics by incorporating a ladder architecture that can infer a hierarchy of dynamical systems. Using some built-in inductive biases for calcium dynamics, we show that we can disentangle calcium flux from the underlying dynamics of neural computation. First, we demonstrate with synthetic calcium data that we can correctly disentangle an underlying Lorenz attractor from calcium dynamics. Next, we show that we can infer appropriate rotational dynamics in spiking data from macaque motor cortex after it has been converted into calcium fluorescence data via a calcium dynamics model. Finally, we show that our method applied to real calcium imaging data from primary visual cortex in mice allows us to infer latent factors that carry salient sensory information about unexpected stimuli. These results demonstrate that variational ladder autoencoders are a promising approach for inferring hierarchical dynamics in experimental settings where the measured variable has its own slow dynamics, such as calcium imaging data. Our new, open-source tool thereby provides the neuroscience community with the ability to apply dynamic latent variable modelling to a wider array of data modalities.
2021-03-07
BioRxiv (preprint)
doi.org
Training a First-Order Theorem Prover from Synthetic Data
Vlad Firoiu
Eser Aygün
Ankit Anand
Zafarali Ahmed
Xavier Glorot
Laurent Orseau
Lei Zhang
Doina Precup
Shibl Mourad
2021-03-04
ArXiv (preprint)
arxiv.org
arxiv.org
Comment on Starke et al.: “Computing schizophrenia: ethical challenges for machine learning in psychiatry”: From machine learning to student learning: pedagogical challenges for psychiatry – Corrigendum
Christophe Gauld
Jean‐Arthur Micoulaud‐Franchi
Guillaume Dumas
2021-03-03
Psychological Medicine (published)
doi.org
Functional specialization within the inferior parietal lobes across cognitive domains
Ole Numssen
Danilo Bzdok
Gesa Hartwigsen
The inferior parietal lobe (IPL) is a key neural substrate underlying diverse mental processes, from basic attention to language and social … (see more)cognition, that define human interactions. Its putative domain-global role appears to tie into poorly understood differences between cognitive domains in both hemispheres. Across attentional, semantic, and social cognitive tasks, our study explored functional specialization within the IPL. The task specificity of IPL subregion activity was substantiated by distinct predictive signatures identified by multivariate pattern-learning algorithms. Moreover, the left and right IPL exerted domain-specific modulation of effective connectivity among their subregions. Task-evoked functional interactions of the anterior and posterior IPL subregions involved recruitment of distributed cortical partners. While anterior IPL subregions were engaged in strongly lateralized coupling links, both posterior subregions showed more symmetric coupling patterns across hemispheres. Our collective results shed light on how under-appreciated hemispheric specialization in the IPL supports some of the most distinctive human mental capacities.
2021-03-01
eLife (published)
doi.org
Integrated inbound train split and load planning in an intermodal railway terminal
Bruno Petrato Bruck
Jean-François Cordeau
Emma Frejinger
2021-02-28
Transportation Research Part B: Methodological (published)
doi.org
Multilevel Approaches for the Critical Node Problem
Andrea Baggio
Margarida Carvalho
Andrea Lodi
Andrea Tramontani
In recent years, a lot of effort has been dedicated to develop strategies to defend networks against possible cascade failures or malicious … (see more)viral attacks. In particular, many results rely on two different viewpoints. On the one hand, network safety is investigated from a preventive perspective. In this paradigm, for a given network, the goal is to modify its structure, in order to minimize the propagation of failures. On the other hand, blocking models have been proposed for scenarios where the attack has already taken place. In this case, a harmful spreading process is assumed to propagate through the network with particular dynamics, allowing some time for an effective defensive reaction. In this work, we combine these two perspectives. More precisely, following the framework Defender-AttackerDefender, we consider a model of prevention, attack, and damage containment using a three-stage, sequential game. Thus, we assume the defender not only to be able to adopt preventive strategies but also to defend the network after an attack takes place. Assuming that the attacker will act optimally, we want to chose a defensive strategy for the first stage that would minimize the total damage to the network in the end of the third stage. Our contribution consists of considering this problem as a trilevel Mixed-Integer Program and design an exact algorithm for it based on tools developed for multilevel programming.
2021-02-28
Operations Research (published)
doi.org
QBSUM: a Large-Scale Query-Based Document Summarization Dataset from Real-world Applications
Mingjun Zhao
Shengli Yan
Bang Liu
Xinwang Zhong
Qian Hao
Haolan Chen
Di Niu
Bo Long
Wei-dong Guo
2021-02-28
Computer Speech & Language (published)
doi.org
arxiv.org
The Locomotive Assignment Problem with Distributed Power at the Canadian National Railway Company
Camilo Ortiz-Astorquiza
Jean-François Cordeau
Emma Frejinger
Some of the most important optimization problems faced by railway operators arise from the management of their locomotive fleet. In this pap… (see more)er, we study a general version of the locomotive assignment problem encountered at the tactical level by one of the largest railroads in North America: the Canadian National Railway Company (CN). We present a modeling framework with two integer linear programming formulations and contribute to the state of the art by allowing to decide each train's operating mode (distributed power or not) over the whole (weekly) planning horizon without partitioning it into smaller time windows. Given the difficulty to solve the problem, one of the formulations is enhanced through various refinements such as constraint relaxations, preprocessing and fixed cost approximations. We thus achieve a significant reduction in the required computational time to solve instances of realistic size. We also present two versions of a Benders decomposition-based algorithm to obtain feasible solutions. On average, it allows to reduce the associated computational time by two hours. Results from an extensive computational study and a case study with data provided by CN confirm the potential benefits of the model and solution approach.
2021-02-28
Transportation Science (published)
doi.org
Towards robust and replicable sex differences in the intrinsic brain function of autism
Dorothea L. Floris
José O. A. Filho
Meng-Chuan Lai
Steve Giavasis
Marianne Oldehinkel
Maarten Mennes
Tony Charman
Julian Tillmann
Guillaume Dumas
Christine Ecker
Flavio Dell’Acqua
Tobias Banaschewski
Carolin Moessnang
Simon Baron-Cohen
Sarah Durston
Eva Loth
Declan Murphy
Jan K. Buitelaar
Christian Beckmann
Michael P. Milham … (see 1 more)
Adriana Di Martino
2021-02-28
Molecular Autism (published)
doi.org
Transformers with Competitive Ensembles of Independent Mechanisms
Alex Lamb
Di He
Anirudh Goyal
Guolin Ke
Chien-Feng Liao
Mirco Ravanelli
Yoshua Bengio
An important development in deep learning from the earliest MLPs has been a move towards architectures with structural inductive biases whic… (see more)h enable the model to keep distinct sources of information and routes of processing well-separated. This structure is linked to the notion of independent mechanisms from the causality literature, in which a mechanism is able to retain the same processing as irrelevant aspects of the world are changed. For example, convnets enable separation over positions, while attention-based architectures (especially Transformers) learn which combination of positions to process dynamically. In this work we explore a way in which the Transformer architecture is deficient: it represents each position with a large monolithic hidden representation and a single set of parameters which are applied over the entire hidden representation. This potentially throws unrelated sources of information together, and limits the Transformer's ability to capture independent mechanisms. To address this, we propose Transformers with Independent Mechanisms (TIM), a new Transformer layer which divides the hidden representation and parameters into multiple mechanisms, which only exchange information through attention. Additionally, we propose a competition mechanism which encourages these mechanisms to specialize over time steps, and thus be more independent. We study TIM on a large-scale BERT model, on the Image Transformer, and on speech enhancement and find evidence for semantically meaningful specialization as well as improved performance.
2021-02-26
ArXiv (preprint)
doi.org
openreview.net
Towards Causal Representation Learning
Bernhard Schölkopf
Francesco Locatello
Stefan Bauer
Nan Rosemary Ke
Nal Kalchbrenner
Anirudh Goyal
Yoshua Bengio
The two fields of machine learning and graphical causality arose and developed separately. However, there is now cross-pollination and incre… (see more)asing interest in both fields to benefit from the advances of the other. In the present paper, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.
2021-02-21
ArXiv (preprint)
arxiv.org
arxiv.org
Model-Invariant State Abstractions for Model-Based Reinforcement Learning
Manan Tomar
Amy Zhang
Roberto Calandra
Matthew E. Taylor
Joelle Pineau
Accuracy and generalization of dynamics models is key to the success of model-based reinforcement learning (MBRL). As the complexity of task… (see more)s increases, so does the sample inefficiency of learning accurate dynamics models. However, many complex tasks also exhibit sparsity in the dynamics, i.e., actions have only a local effect on the system dynamics. In this paper, we exploit this property with a causal invariance perspective in the single-task setting, introducing a new type of state abstraction called \textit{model-invariance}. Unlike previous forms of state abstractions, a model-invariance state abstraction leverages causal sparsity over state variables. This allows for compositional generalization to unseen states, something that non-factored forms of state abstractions cannot do. We prove that an optimal policy can be learned over this model-invariance state abstraction and show improved generalization in a simple toy domain. Next, we propose a practical method to approximately learn a model-invariant representation for complex domains and validate our approach by showing improved modelling performance over standard maximum likelihood approaches on challenging tasks, such as the MuJoCo-based Humanoid. Finally, within the MBRL setting we show strong performance gains with respect to sample efficiency across a host of other continuous control tasks.
2021-02-18
ArXiv (preprint)
openreview.net
openreview.net