Irina Rish

rishika.bhagwatkar@mila.quebec

Biographie

Irina Rish est professeure titulaire à l'Université de Montréal (UdeM), où elle dirige le Laboratoire d'IA autonome. Membre du corps professoral de Mila – Institut québécois d’intelligence artificielle, elle est titulaire d'une chaire d'excellence en recherche du Canada (CERC) et d'une chaire en IA Canada-CIFAR. Irina dirige le projet INCITE du ministère américain de l'Environnement au sujet des modèles de fondation évolutifs sur les superordinateurs Summit et Frontier à l'Oak Ridge Leadership Computing Facility (OLCF). Elle est cofondatrice et directrice scientifique de Nolano.ai.

Ses recherches actuelles portent sur les lois de mise à l'échelle neuronale et les comportements émergents (capacités et alignement) dans les modèles de fondation, ainsi que sur l'apprentissage continu, la généralisation hors distribution et la robustesse. Avant de se joindre à l'UdeM en 2019, Irina était chercheuse au Centre de recherche IBM Thomas J. Watson, où elle a travaillé sur divers projets à l'intersection des neurosciences et de l'IA, et dirigé le défi NeuroAI. Elle a reçu plusieurs prix IBM : ceux de l’excellence et de l’innovation exceptionnelle (2018), celui de la réalisation technique exceptionnelle (2017), et celui de l’accomplissement en recherche (2009). Elle détient 64 brevets et a écrit plus de 120 articles de recherche, plusieurs chapitres de livres, trois livres publiés et une monographie sur la modélisation éparse.

Étudiants actuels

Ivan Anokhin

Doctorat - UdeM

Co-superviseur⋅e :

Samira Ebrahimi Kahou

ivan.anokhin@mila.quebec

Doctorat - UdeM

Arjun Ashok

Doctorat - UdeM

Co-superviseur⋅e :

Alexandre Drouin

arjun.ashok@mila.quebec

Maîtrise recherche - UdeM

Doctorat - UdeM

ethan.caballero@mila.quebec

Pietro Cagnasso

Doctorat - Concordia

Superviseur⋅e principal⋅e :

pietro.cagnasso@mila.quebec

Mohammad Javad Darvishi Bayazi

Amin Darabi

Doctorat - UdeM

amin.darabi@mila.quebec

Collaborateur·rice de recherche - UdeM

mj.darvishi92@gmail.com

nadhir.hassen@mila.quebec

Wagner Drew

Maîtrise recherche - Concordia

Superviseur⋅e principal⋅e :

Mirco Ravanelli

drew.wagner@mila.quebec

Doctorat - UdeM

Visiteur de recherche indépendant - -

parviz.haggi@gmail.com

Nadhir Hassen

Collaborateur·rice de recherche - UdeM

Niklas Herbster

Stagiaire de recherche - UdeM

Co-superviseur⋅e :

Gauthier Gidel

niklas.herbster@gmail.com

prateek.humane@mila.quebec

Mai-Thi Ho

Maîtrise recherche - UdeM

mai-thi.ho@mila.quebec

Prateek Humane

Doctorat

arnav-kumar.jain@mila.quebec

Nizar Islah

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Eilif B. Muller

nizar.islah@mila.quebec

Doctorat - UdeM

Doctorat - UdeM

ayush.kaushal@mila.quebec

Arian Khorasani

Collaborateur·rice de recherche - UdeM

arian.khorasani@mila.quebec

Neeraj Kumar

Collaborateur·rice alumni - UdeM

neeraj.kumar@mila.quebec

Gwen Legate

Doctorat - Concordia

Superviseur⋅e principal⋅e :

gwendolyne.legate@mila.quebec

David Lemay

Maîtrise recherche - UdeM

david.lemay@mila.quebec

amin.mansouri@mila.quebec

Amin Mansouri

Collaborateur·rice alumni - UdeM

Collaborateur·rice de recherche

Doctorat - UdeM

Collaborateur·rice de recherche - UdeM

diganta.misra@mila.quebec

motahareh.pourrahimi@mila.quebec

Gabriela Moisescu-Pareja

Collaborateur·rice de recherche - McGill

Superviseur⋅e principal⋅e :

Doina Precup

moisescg@mila.quebec

Timothy Nest

Doctorat - UdeM

Co-superviseur⋅e :

Eilif B. Muller

timothy.nest@mila.quebec

Jonathan Pilault

Collaborateur·rice de recherche - Polytechnique

Doctorat - McGill

Superviseur⋅e principal⋅e :

Pouya Bashivan

Mahta Ramezanian

Maîtrise recherche - UdeM

Co-superviseur⋅e :

Guillaume Dumas

mahta.ramezanian@mila.quebec

Matthew Riemer

Doctorat - UdeM

matthew.riemer@mila.quebec

Alexis Roger

Doctorat - McGill

Superviseur⋅e principal⋅e :

Blake Richards

alexis.roger@mila.quebec

munish.sathish-kumar@mila.quebec

Munish Sathish Kumar

Collaborateur·rice de recherche

Zibo Shang

Doctorat - UdeM

zibo.shang@mila.quebec

Vaibhav Singh

Doctorat - Concordia

Superviseur⋅e principal⋅e :

vaibhav.singh@mila.quebec

Gopeshh Subbaraj

Doctorat - UdeM

gopeshh.subbaraj@mila.quebec

Doctorat - UdeM

Co-superviseur⋅e :

benjamin.therien@mila.quebec

Collaborateur·rice de recherche - UdeM

tejas.vaidhya@mila.quebec

andrew.williams@mila.quebec

Sihui Wei

Baccalauréat - McGill

sihui.wei@mila.quebec

Andrew Williams

Doctorat - UdeM

Co-superviseur⋅e :

Nicolas Chapados

daria.yasafova@mila.quebec

Daria Yasafova

Maîtrise recherche - UdeM

He Zhu

Doctorat - McGill

he.zhu@mila.quebec

Publications

Optimizing deep learning for Magnetoencephalography (MEG): From sensory perception to sex prediction and brain fingerprinting

Arthur Dehgan

Jean-christophe Gagnon-audet

Karim Jerbi

2021-12-31

2022 Conference on Cognitive Computational Neuroscience (publié)

PRACTICAL GUIDE

Paolo Bellavista

2021-12-31

(publié)

www.semanticscholar.org

Generative Models of Brain Dynamics -- A review

Mahta Ramezanian Panahi

Germán Abrevaya

Vikram Voleti

Guillaume Dumas

The principled design and discovery of biologically- and physically-informed models of neuronal dynamics has been advancing since the mid-tw… (voir plus)entieth century. Recent developments in artificial intelligence (AI) have accelerated this progress. This review article gives a high-level overview of the approaches across different scales of organization and levels of abstraction. The studies covered in this paper include fundamental models in computational neuroscience, nonlinear dynamics, data-driven methods, as well as emergent practices. While not all of these models span the intersection of neuroscience, AI, and system dynamics, all of them do or can work in tandem as generative models, which, as we argue, provide superior properties for the analysis of neuroscientific data. We discuss the limitations and unique dynamical traits of brain data and the complementary need for hypothesis- and data-driven modeling. By way of conclusion, we present several hybrid generative models from recent literature in scientific machine learning, which can be efficiently deployed to yield interpretable models of neural dynamics.

2021-12-21

ArXiv (prépublication)

arxiv.org

Adversarial Feature Desensitization

Neural networks are known to be vulnerable to adversarial attacks -- slight but carefully constructed perturbations of the inputs which can … (voir plus)drastically impair the network's performance. Many defense methods have been proposed for improving robustness of deep networks by training them on adversarially perturbed inputs. However, these models often remain vulnerable to new types of attacks not seen during training, and even to slightly stronger versions of previously seen attacks. In this work, we propose a novel approach to adversarial robustness, which builds upon the insights from the domain adaptation field. Our method, called Adversarial Feature Desensitization (AFD), aims at learning features that are invariant towards adversarial perturbations of the inputs. This is achieved through a game where we learn features that are both predictive and robust (insensitive to adversarial attacks), i.e. cannot be used to discriminate between natural and adversarial data. Empirical results on several benchmarks demonstrate the effectiveness of the proposed approach against a wide range of attack types and attack strengths. Our code is available at https://github.com/BashivanLab/afd.

2021-12-06

Conference on Neural Information Processing Systems (Accept (Poster))

Jean-christophe Gagnon-audet

Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization

Kartik Ahuja

Ethan Caballero

Dinghuai Zhang

Yoshua Bengio

Ioannis Mitliagkas

The invariance principle from causality is at the heart of notable approaches such as invariant risk minimization (IRM) that seek to address… (voir plus) out-of-distribution (OOD) generalization failures. Despite the promising theory, invariance principle-based approaches fail in common classification tasks, where invariant (causal) features capture all the information about the label. Are these failures due to the methods failing to capture the invariance? Or is the invariance principle itself insufficient? To answer these questions, we revisit the fundamental assumptions in linear regression tasks, where invariance-based approaches were shown to provably generalize OOD. In contrast to the linear regression tasks, we show that for linear classification tasks we need much stronger restrictions on the distribution shifts, or otherwise OOD generalization is impossible. Furthermore, even with appropriate restrictions on distribution shifts in place, we show that the invariance principle alone is insufficient. We prove that a form of the information bottleneck constraint along with invariance helps address key failures when invariant features capture all the information about the label and also retains the existing success when they do not. We propose an approach that incorporates both of these principles and demonstrate its effectiveness in several experiments.

2021-12-06

Conference on Neural Information Processing Systems (Accept (Spotlight))

Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers

Empirical science of neural scaling laws is a rapidly growing area of significant importance to the future of machine learning, particularly… (voir plus) in the light of recent breakthroughs achieved by large-scale pre-trained models such as GPT-3, CLIP and DALL-e. Accurately predicting the neural network performance with increasing resources such as data, compute and model size provides a more comprehensive evaluation of different approaches across multiple scales, as opposed to traditional point-wise comparisons of fixed-size models on fixed-size benchmarks, and, most importantly, allows for focus on the best-scaling, and thus most promising in the future, approaches. In this work, we consider a challenging problem of few-shot learning in image classification, especially when the target data distribution in the few-shot phase is different from the source, training, data distribution, in a sense that it includes new image classes not encountered during training. Our current main goal is to investigate how the amount of pre-training data affects the few-shot generalization performance of standard image classifiers. Our key observations are that (1) such performance improvements are well-approximated by power laws (linear log-log plots) as the training set size increases, (2) this applies to both cases of target data coming from either the same or from a different domain (i.e., new classes) as the training data, and (3) few-shot performance on new classes converges at a faster rate than the standard classification performance on previously seen classes. Our findings shed new light on the relationship between scale and generalization.

2021-10-12

arXiv (prépublication)

Approximate Bayesian Optimisation for Neural Networks

Nadhir Hassen

2021-08-26

ArXiv (prépublication)

arxiv.org

Toward Optimal Solution for the Context-Attentive Bandit Problem

Djallel Bouneffouf

Raphael Feraud

Sohini Upadhyay

Yasaman Khazaeni

2021-08-18

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (publié)

Sequoia: A Software Framework to Unify Continual Learning Research

Pau Rodríguez

J. Hurtado

Dominic Zhao

Ryan Lindeborg

Timothee LESORT

David Vázquez

Laurent Charlin

Massimo Caccia

The field of Continual Learning (CL) seeks to develop algorithms that accumulate knowledge and skills over time through interaction with non… (voir plus)-stationary environments. In practice, a plethora of evaluation procedures (settings) and algorithmic solutions (methods) exist, each with their own potentially disjoint set of assumptions. This variety makes measuring progress in CL difficult. We propose a taxonomy of settings, where each setting is described as a set of assumptions. A tree-shaped hierarchy emerges from this view, where more general settings become the parents of those with more restrictive assumptions. This makes it possible to use inheritance to share and reuse research, as developing a method for a given setting also makes it directly applicable onto any of its children. We instantiate this idea as a publicly available software framework called Sequoia, which features a wide variety of settings from both the Continual Supervised Learning (CSL) and Continual Reinforcement Learning (CRL) domains. Sequoia also includes a growing suite of methods which are easy to extend and customize, in addition to more specialized methods from external libraries. We hope that this new paradigm and its first implementation can help unify and accelerate research in CL. You can help us grow the tree by visiting www.github.com/lebrice/Sequoia.

2021-08-01

ArXiv (prépublication)

Jean-christophe Gagnon-audet

Learning Brain Dynamics With Coupled Low-Dimensional Nonlinear Oscillators and Deep Recurrent Networks.

Germán Abrevaya

Guillaume Dumas

Aleksandr Y. Aravkin

Peng Zheng

James R. Kozloski

Pablo Polosecki

Guillaume Lajoie

David D. Cox

Silvina Ponce Dawson

Guillermo A. Cecchi

Many natural systems, especially biological ones, exhibit complex multivariate nonlinear dynamical behaviors that can be hard to capture by … (voir plus)linear autoregressive models. On the other hand, generic nonlinear models such as deep recurrent neural networks often require large amounts of training data, not always available in domains such as brain imaging; also, they often lack interpretability. Domain knowledge about the types of dynamics typically observed in such systems, such as a certain type of dynamical systems models, could complement purely data-driven techniques by providing a good prior. In this work, we consider a class of ordinary differential equation (ODE) models known as van der Pol (VDP) oscil lators and evaluate their ability to capture a low-dimensional representation of neural activity measured by different brain imaging modalities, such as calcium imaging (CaI) and fMRI, in different living organisms: larval zebrafish, rat, and human. We develop a novel and efficient approach to the nontrivial problem of parameters estimation for a network of coupled dynamical systems from multivariate data and demonstrate that the resulting VDP models are both accurate and interpretable, as VDP's coupling matrix reveals anatomically meaningful excitatory and inhibitory interactions across different brain subsystems. VDP outperforms linear autoregressive models (VAR) in terms of both the data fit accuracy and the quality of insight provided by the coupling matrices and often tends to generalize better to unseen data when predicting future brain activity, being comparable to and sometimes better than the recurrent neural networks (LSTMs). Finally, we demonstrate that our (generative) VDP model can also serve as a data-augmentation tool leading to marked improvements in predictive accuracy of recurrent neural networks. Thus, our work contributes to both basic and applied dimensions of neuroimaging: gaining scientific insights and improving brain-based predictive models, an area of potentially high practical importance in clinical diagnosis and neurotechnology.

2021-06-06

Neural Computation (inconnu)

Double-Linear Thompson Sampling for Context-Attentive Bandits

Djallel Bouneffouf

Raphael Feraud

Sohini Upadhyay

Yasaman Khazaeni

In this paper, we analyze and extend an online learning frame-work known as Context-Attentive Bandit, motivated by various practical applica… (voir plus)tions, from medical diagnosis to dialog systems, where due to observation costs only a small subset of a potentially large number of context variables can be observed at each iteration; however, the agent has a freedom to choose which variables to observe. We derive a novel algorithm, called Context-Attentive Thompson Sampling (CATS), which builds upon the Linear Thompson Sampling approach, adapting it to Context-Attentive Bandit setting. We provide a theoretical regret analysis and an extensive empirical evaluation demonstrating advantages of the proposed approach over several baseline methods on a variety of real-life datasets.

2021-06-05

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (publié)

arxiv.org

Toward Skills Dialog Orchestration with Online Learning

Djallel Bouneffouf

Raphael Feraud

Sohini Upadhyay

Mayank Agarwal

Yasaman Khazaeni

Building multi-domain AI agents is a challenging task and an open problem in the area of AI. Within the domain of dialog, the ability to orc… (voir plus)hestrate multiple independently trained dialog agents, or skills, to create a unified system is of particular significance. In this work, we study the task of online posterior dialog orchestration, where we define posterior orchestration as the task of selecting a subset of skills which most appropriately answer a user input using features extracted from both the user input and the individual skills. To account for the various costs associated with extracting skill features, we consider online posterior orchestration under a skill execution budget. We formalize this setting as Context Attentive Bandit with Observations (CABO), a variant of context attentive bandits, and evaluate it on proprietary conversational datasets.

2021-06-05

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (publié)