Publications

A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix
Thang Doan
Mehdi Abbana Bennani
Bogdan Mazoure
Guillaume Rabusseau
Pierre Alquier
2021-01-01
AISTATS (published)
proceedings.mlr.press
arxiv.org
Towards a Trace-Preserving Tensor Network Representation of Quantum Channels
Siddarth Srinivasan
Sandesh M. Adhikary
Jacob Miller
Bibek Pokharel
Guillaume Rabusseau
Byron Boots
The problem of characterizing quantum channels arises in a number of contexts such as quantum process tomography and quantum error correctio… (see more)n. However, direct approaches to parameterizing and optimizing the Choi matrix representation of quantum channels face a curse of dimensionality: the number of parameters scales exponentially in the number of qubits. Recently, Torlai et al. [2020] proposed using locally purified density operators (LPDOs), a tensor network representation of Choi matrices, to overcome the unfavourable scaling in parameters. While the LPDO structure allows it to satisfy a ‘complete positivity’ (CP) constraint required of physically valid quantum channels, it makes no guarantees about a similarly required ‘trace preservation’ (TP) constraint. In practice, the TP constraint is violated, and the learned quantum channel may even be trace-increasing, which is non-physical. In this work, we present the problem of optimizing over TP LPDOs, discuss two approaches to characterizing the TP constraints on LPDOs, and outline the next steps for developing an optimization scheme.
2021-01-01
(published)
www.semanticscholar.org
A Unified Few-Shot Classification Benchmark to Compare Transfer and Meta Learning Approaches
Vincent Dumoulin
Neil Houlsby
Utku Evci
Xiaohua Zhai
Ross Goroshin
Sylvain Gelly
Hugo Larochelle
Meta and transfer learning are two successful families of approaches to few-shot 1 learning. Despite highly related goals, state-of-the-art … (see more)advances in each family are 2 measured largely in isolation of each other. As a result of diverging evaluation 3 norms, a direct or thorough comparison of different approaches is challenging. 4 To bridge this gap, we introduce a few-shot classification evaluation protocol 5 named VTAB+MD with the explicit goal of facilitating sharing of insights from 6 each community. We demonstrate its accessibility in practice by performing a 7 cross-family study of the best transfer and meta learners which report on both a 8 large-scale meta-learning benchmark (Meta-Dataset, MD), and a transfer learning 9 benchmark (Visual Task Adaptation Benchmark, VTAB). We find that, on average, 10 large-scale transfer methods (Big Transfer, BiT) outperform competing approaches 11 on MD, even when trained only on ImageNet. In contrast, meta-learning approaches 12 struggle to compete on VTAB when trained and validated on MD. However, BiT 13 is not without limitations, and pushing for scale does not improve performance 14 on highly out-of-distribution MD tasks. We hope that this work contributes to 15 accelerating progress on few-shot learning research. 16
2021-01-01
NeurIPS Datasets and Benchmarks (published)
openreview.net
openreview.net
A Universal Representation Transformer Layer for Few-Shot Image Classification
Lu Liu
William Hamilton
Guodong Long
Jing Jiang
Hugo Larochelle
Few-shot classification aims to recognize unseen classes when presented with only a small number of samples. We consider the problem of mult… (see more)i-domain few-shot image classification, where unseen classes and examples come from diverse data sources. This problem has seen growing interest and has inspired the development of benchmarks such as Meta-Dataset. A key challenge in this multi-domain setting is to effectively integrate the feature representations from the diverse set of training domains. Here, we propose a Universal Representation Transformer (URT) layer, that meta-learns to leverage universal features for few-shot classification by dynamically re-weighting and composing the most appropriate domain-specific representations. In experiments, we show that URT sets a new state-of-the-art result on Meta-Dataset. Specifically, it achieves top-performance on the highest number of data sources compared to competing methods. We analyze variants of URT and present a visualization of the attention score heatmaps that sheds light on how the model performs cross-domain generalization.
2021-01-01
ICLR (published)
openreview.net
openreview.net
Your head is there to move you around: Goal-driven models of the primate dorsal pathway
Patrick J Mineault
Shahab Bakhtiari
Blake Richards
Christopher C. Pack
Neurons in the dorsal visual pathway of the mammalian brain are selective for motion stimuli, with the complexity of stimulus representation… (see more)s increasing along the hierarchy. This progression is similar to that of the ventral visual pathway, which is well characterized by artificial neural networks (ANNs) optimized for object recognition. In contrast, there are no image-computable models of the dorsal stream with comparable explanatory power. We hypothesized that the properties of dorsal stream neurons could be explained by a simple learning objective: the need for an organism to orient itself during self-motion. To test this hypothesis, we trained a 3D ResNet to predict an agent’s self-motion parameters from visual stimuli in a simulated environment. We found that the responses in this network accounted well for the selectivity of neurons in a large database of single-neuron recordings from the dorsal visual stream of non-human primates. In contrast, ANNs trained on an action recognition dataset through supervised or self-supervised learning could not explain responses in the dorsal stream, despite also being trained on naturalistic videos with moving objects. These results demonstrate that an ecologically relevant cost function can account for dorsal stream properties in the primate brain.
openreview.net
Mass‐spectrometry analysis of the human pineal proteome during night and day and in autism
Guillaume Dumas
Hany Goubran‐Botros
Mariette Matondo
Cécile Pagan
Cyril Boulègue
Thibault Chaze
Julia Chamot‐Rooke
Erik Maronde
Thomas Bourgeron
The human pineal gland regulates day‐night dynamics of multiple physiological processes, especially through the secretion of melatonin. Us… (see more)ing mass‐spectrometry‐based proteomics and dedicated analysis tools, we identify proteins in the human pineal gland and analyze systematically their variation throughout the day and compare these changes in the pineal proteome between control specimens and donors diagnosed with autism. Results reveal diverse regulated clusters of proteins with, among others, catabolic carbohydrate process and cytoplasmic membrane‐bounded vesicle‐related proteins differing between day and night and/or control versus autism pineal glands. These data show novel and unexpected processes happening in the human pineal gland during the day/night rhythm as well as specific differences between autism donor pineal glands and those from controls.
2020-12-23
Journal of Pineal Research (published)
doi.org
Phylogenetic Manifold Regularization: A semi-supervised approach to predict transcription factor binding sites
Faizy Ahsan
Alexandre Drouin
François Laviolette
Doina Precup
Mathieu Blanchette
The computational prediction of transcription factor binding sites remains a challenging problems in bioinformatics, despite significant met… (see more)hodological developments from the field of machine learning. Such computational models are essential to help interpret the non-coding portion of human genomes, and to learn more about the regulatory mechanisms controlling gene expression. In parallel, massive genome sequencing efforts have produced assembled genomes for hundred of vertebrate species, but this data is underused. We present PhyloReg, a new semi-supervised learning approach that can be used for a wide variety of sequence-to-function prediction problems, and that takes advantage of hundreds of millions of years of evolution to regularize predictors and improve accuracy. We demonstrate that PhyloReg can be used to better train a previously proposed deep learning model of transcription factor binding. Simulation studies further help delineate the benefits of the a pproach. G ains in prediction accuracy are obtained over a broad set of transcription factors and cell types.
2020-12-16
2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (published)
doi.org
Assessing the Electronic Evidence System Needs of Canadian Public Health Professionals: A Cross-Sectional Study (Preprint)
Bandna Dhaliwal
Sarah E Neil-Sztramko
Nikita Boston-Fisher
David Buckeridge
Maureen Dobbins
BACKGROUND True evidence-informed decision making in public health relies on incorporating evidence from a number of sources in addition to… (see more) traditional scientific evidence. Lack of access to these types of data, as well as ease of use and interpretability of scientific evidence contribute to limited uptake of evidence-informed decision making in practice. An electronic evidence system that includes multiple sources of evidence and potentially novel computational processing approaches or artificial intelligence holds promise as a solution to overcoming barriers to evidence-informed decision making in public health. OBJECTIVE To understand the needs and preferences for an electronic evidence system among public health professionals in Canada. METHODS An invitation to participate in an anonymous online survey was distributed via listservs of two Canadian public health organizations. Eligible participants were English or French speaking individuals currently working in public health. The survey contained both multiple choice and open-ended questions about needs and preferences relevant to an electronic evidence system. Quantitative responses were analyzed to explore differences by public health role. Inductive and deductive analysis methods were used to code and interpret the qualitative data. Ethics review was not required by the host institution. RESULTS Respondents (n = 371) were heterogeneous, spanning organizations, positions, and areas of practice within public health. Nearly all (98.0%) respondents indicated that an electronic evidence system would support their work. Respondents had high preferences for local contextual data, research and intervention evidence, and information about human and financial resources. Qualitative analyses identified a number of concerns, needs, and suggestions for development of such a system. Concerns ranged from personal use of such a system, to the ability of their organization to use such a system. Identified needs spanned the different sources of evidence including local context, research and intervention evidence, and resources and tools. Additional suggestions were identified to improve system usability. CONCLUSIONS Canadian public health professionals have positive perceptions towards an electronic evidence system that would bring together evidence from the local context, scientific research, and resources. Elements were also identified to increase the usability of an electronic evidence system.
2020-12-14
JMIR Public Health and Surveillance (published)
doi.org
Graph Neural Networks Learn Twitter Bot Behaviour
Albert Manuel Orozco Camacho
Sacha Lévy
Reihaneh Rabbany
Social media trends are increasingly taking a significant role for the understanding of modern social dynamics. In this work, we take a look… (see more) at how the Twitter landscape gets constantly shaped by automatically generated content. Twitter bot activity can be traced via network abstractions which, we hypothesize, can be learned through state-of-the-art graph neural network techniques. We employ a large bot database, continuously updated by Twitter, to learn how likely is that a user is mentioned by a bot, as well as, for a hashtag. Thus, we model this likelihood as a link prediction task between the set of users and hashtags. Moreover, we contrast our results by performing similar experiments on a crawled data set of real users.
2020-12-12
LatinX in AI at Neural Information Processing Systems Conference 2020 (published)
doi.org
Extendable and invertible manifold learning with geometry regularized autoencoders
Andres F. Duque Correa
Sacha Morin
Guy Wolf
Kevin R. Moon
A fundamental task in data exploration is to extract simplified low dimensional representations that capture intrinsic geometry in data, esp… (see more)ecially for faithfully visualizing data in two or three dimensions. Common approaches to this task use kernel methods for manifold learning. However, these methods typically only provide an embedding of fixed input data and cannot extend to new data points. Autoencoders have also recently become popular for representation learning. But while they naturally compute feature extractors that are both extendable to new data and invertible (i.e., reconstructing original features from latent representation), they have limited capabilities to follow global intrinsic geometry compared to kernel-based manifold learning. We present a new method for integrating both approaches by incorporating a geometric regularization term in the bottleneck of the autoencoder. Our regularization, based on the diffusion potential distances from the recently-proposed PHATE visualization method, encourages the learned latent representation to follow intrinsic data geometry, similar to manifold learning algorithms, while still enabling faithful extension to new data and reconstruction of data in the original feature space from latent coordinates. We compare our approach with leading kernel methods and autoencoder models for manifold learning to provide qualitative and quantitative evidence of our advantages in preserving intrinsic structure, out of sample extension, and reconstruction. Our method is easily implemented for big-data applications, whereas other methods are limited in this regard.
2020-12-10
2020 IEEE International Conference on Big Data (Big Data) (published)
doi.org
arxiv.org
A Study of Condition Numbers for First-Order Optimization
Charles Guille-Escuret
Baptiste Goujaud
Manuela Girotti
Ioannis Mitliagkas
The study of first-order optimization algorithms (FOA) typically starts with assumptions on the objective functions, most commonly smoothnes… (see more)s and strong convexity. These metrics are used to tune the hyperparameters of FOA. We introduce a class of perturbations quantified via a new norm, called *-norm. We show that adding a small perturbation to the objective function has an equivalently small impact on the behavior of any FOA, which suggests that it should have a minor impact on the tuning of the algorithm. However, we show that smoothness and strong convexity can be heavily impacted by arbitrarily small perturbations, leading to excessively conservative tunings and convergence issues. In view of these observations, we propose a notion of continuity of the metrics, which is essential for a robust tuning strategy. Since smoothness and strong convexity are not continuous, we propose a comprehensive study of existing alternative metrics which we prove to be continuous. We describe their mutual relations and provide their guaranteed convergence rates for the Gradient Descent algorithm accordingly tuned. Finally we discuss how our work impacts the theoretical understanding of FOA and their performances.
2020-12-10
ArXiv (preprint)
arxiv.org
arxiv.org
Team Optimal Control of Coupled Major-Minor Subsystems with Mean-Field Sharing
Jalal Arabneydi
Aditya Mahajan
2020-12-04
ArXiv (preprint)
arxiv.org
arxiv.org