Doina Precup

busra-tugce.gurbuz@mila.quebec

Jesse Farebrother

Doctorat - McGill

Superviseur⋅e principal⋅e :

Marc Gendron-Bellemare

Doctorat - McGill

Superviseur⋅e principal⋅e :

Eilif B. Muller

Mohammad Sami Nur Islam Islam

Jonathan Hu

Maîtrise recherche - McGill

jonathan.hu@mila.quebec

Howard Huang

Doctorat - McGill

huanghow@mila.quebec

Maîtrise recherche - McGill

mohammad-sami-nur.islam@mila.quebec

Hangzhan Jin

Doctorat - Polytechnique

jinhangzhan@gmail.com

flemming.kondrup@mila.quebec

Flemming Kondrup

Postdoctorat - McGill

Jonathan Lebensold

Collaborateur·rice alumni - McGill

Collaborateur·rice alumni - McGill

jeremy.lucas@mila.quebec

Ray Luo

Doctorat - McGill

Superviseur⋅e principal⋅e :

Xujie Si

luo.ziyan@mila.quebec

Sicheng Lyu

Maîtrise recherche - McGill

sicheng.lyu@mila.quebec

G McCracken

Doctorat - McGill

mccrackg@mila.quebec

shahrad.mohammadzadeh@mila.quebec

Shahrad Mohammadzadeh

Maîtrise recherche - McGill

Superviseur⋅e principal⋅e :

Reihaneh Rabbany

padideh.nouri@mila.quebec

Gabriela Moisescu-Pareja

Collaborateur·rice de recherche - McGill

Co-superviseur⋅e :

Irina Rish

moisescg@mila.quebec

Padideh Nouri

Doctorat - UdeM

Doctorat - McGill

Co-superviseur⋅e :

Stagiaire de recherche - McGill

william.potter@mila.quebec

manoosh.samiei@mila.quebec

Nate Rahn

Doctorat - McGill

Superviseur⋅e principal⋅e :

Marc Gendron-Bellemare

nathan.rahn@mila.quebec

Mandana Samiei

Doctorat - McGill

Co-superviseur⋅e :

Blake Richards

samiemandana@gmail.com

Doctorat - McGill

Co-superviseur⋅e :

Paul Masset

Doctorat - McGill

Nishanth Anand Vemgal

Doctorat - McGill

Doctorat - McGill

Co-superviseur⋅e :

Samira Ebrahimi Kahou

Stagiaire de recherche - McGill

guangyuan.wang@mila.quebec

Zihan Wang

Doctorat - McGill

zihan.wang@mila.quebec

shuyuan.zhang@mila.quebec

Steve Wen

Maîtrise recherche - McGill

Co-superviseur⋅e :

Gregory Dudek

steve.wen@mila.quebec

Zijing Wu

Doctorat - McGill

Superviseur⋅e principal⋅e :

Paul Masset

zijing.wu@mila.quebec

Doctorat - McGill

Harry Zhao

Collaborateur·rice alumni - McGill

Co-superviseur⋅e :

Billets de blogue

Generic thumbnail for Mila Blog articles.

22 février 2024

Skipper : combiner l’abstraction spatiale et temporelle afin d’améliorer la généralisation

par

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Lire l'article

Publications

Temporal Abstraction in Reinforcement Learning with the Successor Representation

Marlos C. Machado

Andre Barreto

Michael Bowling

2022-12-31

J. Mach. Learn. Res. (publié)

arxiv.org

Assessing Intrapartum Risk of Hypoxic-Ischemic Encephalopathy using Fetal Heart Rate with Long Short-term Memory Networks

"Derek Kweku DEGBEDZUI

Michael Kuzniewicz

Cornet Marie-Coralie

Yvonne Wu

Heather Forquer

Lawrence Gerstley

Emily Hamilton

Philip Warrick

Robert Kearney"

This study investigated the prediction of the risk of hypoxic ischemic encephalopathy using intrapartum cardiotocography records with a long… (voir plus) short-term memory re-current neural network. Across the 12 hours of labour, HIE sensitivity rose from 0.25 to 0.56 as delivery approached while specificity remained approximately constant with a mean of 0.71 and standard deviation of 0.04. The results show that classification improves as delivery approaches but that performance needs improvement. Future work will address the limitations of this preliminary study by investigating input signal transformations and the use of other network architectures to improve the model performance.

2022-12-30

Computing in Cardiology Conference (CinC) (publié)

Offline Policy Optimization in RL with Variance Regularizaton

Riashat Islam

Samarth Sinha

Homanga Bharadhwaj

Samin Yeasar Arnob

Zhuoran Yang

Animesh Garg

Zhaoran Wang

Lihong Li

2022-12-28

arXiv (prépublication)

arxiv.org

Towards Continual Reinforcement Learning: A Review and Perspectives

Khimya Khetarpal

Matthew D Riemer

Irina Rish

2022-12-21

Journal of Artificial Intelligence Research (publié)

arxiv.org

Bayesian Q-learning With Imperfect Expert Demonstrations

Xiru Zhu

Guided exploration with expert demonstrations improves data efficiency for reinforcement learning, but current algorithms often overuse expe… (voir plus)rt information. We propose a novel algorithm to speed up Q-learning with the help of a limited amount of imperfect expert demonstrations. The algorithm avoids excessive reliance on expert data by relaxing the optimal expert assumption and gradually reducing the usage of uninformative expert data. Experimentally, we evaluate our approach on a sparse-reward chain environment and six more complicated Atari games with delayed rewards. With the proposed methods, we can achieve better results than Deep Q-learning from Demonstrations (Hester et al., 2017) in most environments.

2022-12-08

NeurIPS.cc/2022/Workshop/DeepRL (accepté)

Continuous MDP Homomorphisms and Homomorphic Policy Gradient

Sahand Rezaei-Shoshtari

Abstraction has been widely studied as a way to improve the efficiency and generalization of reinforcement learning algorithms. In this pape… (voir plus)r, we study abstraction in the continuous-control setting. We extend the definition of MDP homomorphisms to encompass continuous actions in continuous state spaces. We derive a policy gradient theorem on the abstract MDP, which allows us to leverage approximate symmetries of the environment for policy optimization. Based on this theorem, we propose an actor-critic algorithm that is able to learn the policy and the MDP homomorphism map simultaneously, using the lax bisimulation metric. We demonstrate the effectiveness of our method on benchmark tasks in the DeepMind Control Suite. Our method's ability to utilize MDP homomorphisms for representation learning leads to improved performance when learning from pixel observations.

2022-11-28

Neural Information Processing Systems (Accept)

Revisiting Heterophily For Graph Neural Networks

Sitao Luan

Chenqing Hua

Qincheng Lu

Jiaqi Zhu

Mingde Zhao

Shuyuan Zhang

Xiao-Wen Chang

Graph Neural Networks (GNNs) extend basic Neural Networks (NNs) by using graph structures based on the relational inductive bias (homophily … (voir plus)assumption). While GNNs have been commonly believed to outperform NNs in real-world tasks, recent work has identified a non-trivial set of datasets where their performance compared to NNs is not satisfactory. Heterophily has been considered the main cause of this empirical observation and numerous works have been put forward to address it. In this paper, we first revisit the widely used homophily metrics and point out that their consideration of only graph-label consistency is a shortcoming. Then, we study heterophily from the perspective of post-aggregation node similarity and define new homophily metrics, which are potentially advantageous compared to existing ones. Based on this investigation, we prove that some harmful cases of heterophily can be effectively addressed by local diversification operation. Then, we propose the Adaptive Channel Mixing (ACM), a framework to adaptively exploit aggregation, diversification and identity channels node-wisely to extract richer localized information for diverse node heterophily situations. ACM is more powerful than the commonly used uni-channel framework for node classification tasks on heterophilic graphs and is easy to be implemented in baseline GNN layers. When evaluated on 10 benchmark node classification tasks, ACM-augmented baselines consistently achieve significant performance gain, exceeding state-of-the-art GNNs on most tasks without incurring significant computational burden.

2022-11-28

Conference on Neural Information Processing Systems (Accept)

Complete the Missing Half: Augmenting Aggregation Filtering with Diversification for Graph Convolutional Networks

Mingde Zhao

Xiao-Wen Chang

The core operation of current Graph Neural Networks (GNNs) is the aggregation enabled by the graph Laplacian or message passing, which filte… (voir plus)rs the neighborhood node information. Though effective for various tasks, in this paper, we show that they are potentially a problematic factor underlying all GNN methods for learning on certain datasets, as they force the node representations similar, making the nodes gradually lose their identity and become indistinguishable. Hence, we augment the aggregation operations with their dual, i.e. diversification operators that make the node more distinct and preserve the identity. Such augmentation replaces the aggregation with a two-channel filtering process that, in theory, is beneficial for enriching the node representations. In practice, the proposed two-channel filters can be easily patched on existing GNN methods with diverse training strategies, including spectral and spatial (message passing) methods. In the experiments, we observe desired characteristics of the models and significant performance boost upon the baselines on 9 node classification tasks.

2022-11-21

NeurIPS.cc/2022/Workshop/GLFrontiers (accepté)

Behind the Machine's Gaze: Neural Networks with Biologically-inspired Constraints Exhibit Human-like Visual Attention

Leo Schwinn

Bjoern Eskofier

Dario Zanca

By and large, existing computational models of visual attention tacitly assume perfect vision and full access to the stimulus and thereby de… (voir plus)viate from foveated biological vision. Moreover, modeling top-down attention is generally reduced to the integration of semantic features without incorporating the signal of a high-level visual tasks that have been shown to partially guide human attention. We propose the Neural Visual Attention (NeVA) algorithm to generate visual scanpaths in a top-down manner. With our method, we explore the ability of neural networks on which we impose a biologically-inspired foveated vision constraint to generate human-like scanpaths without directly training for this objective. The loss of a neural network performing a downstream visual task (i.e., classification or reconstruction) flexibly provides top-down guidance to the scanpath. Extensive experiments show that our method outperforms state-of-the-art unsupervised human attention models in terms of similarity to human scanpaths. Additionally, the flexibility of the framework allows to quantitatively investigate the role of different tasks in the generated visual behaviors. Finally, we demonstrate the superiority of the approach in a novel experiment that investigates the utility of scanpaths in real-world applications, where imperfect viewing conditions are given.

2022-10-29

TMLR (accepté)

Simulating Human Gaze with Neural Visual Attention

Leo Schwinn

Bjoern Eskofier

Dario Zanca

2022-10-19

NeurIPS.cc/2022/Workshop/GMML (présentation orale)

The Paradox of Choice: On the Role of Attention in Hierarchical Reinforcement Learning

Andrei Cristian Nica

Khimya Khetarpal

Decision-making AI agents are often faced with two important challenges: the depth of the planning horizon, and the branching factor due to … (voir plus)having many choices. Hierarchical reinforcement learning methods aim to solve the first problem, by providing shortcuts that skip over multiple time steps. To cope with the breadth, it is desirable to restrict the agent's attention at each step to a reasonable number of possible choices. The concept of affordances (Gibson, 1977) suggests that only certain actions are feasible in certain states. In this work, we first characterize "affordances" as a "hard" attention mechanism that strictly limits the available choices of temporally extended options. We then investigate the role of hard versus soft attention in training data collection, abstract value learning in long-horizon tasks, and handling a growing number of choices. To this end, we present an online, model-free algorithm to learn affordances that can be used to further learn subgoal options. Finally, we identify and empirically demonstrate the settings in which the "paradox of choice" arises, i.e. when having fewer but more meaningful choices improves the learning speed and performance of a reinforcement learning agent.

2022-10-19

NeurIPS.cc/2022/Workshop/Attention (poster)

Estimating individual treatment effect on disability progression in multiple sclerosis using deep learning

Jean-Pierre R. Falet

Joshua Durso-finley

Brennan Nichyporuk

Julien Schroeter

Francesca Bovis

Maria-Pia Sormani

Tal Arbel

Douglas Lorne Arnold

Disability progression in multiple sclerosis remains resistant to treatment. The absence of a suitable biomarker to allow for phase 2 clinic… (voir plus)al trials presents a high barrier for drug development. We propose to enable short proof-of-concept trials by increasing statistical power using a deep-learning predictive enrichment strategy. Specifically, a multi-headed multilayer perceptron is used to estimate the conditional average treatment effect (CATE) using baseline clinical and imaging features, and patients predicted to be most responsive are preferentially randomized into a trial. Leveraging data from six randomized clinical trials ( n = 3,830), we first pre-trained the model on the subset of relapsing-remitting MS patients ( n = 2,520), then fine-tuned it on a subset of primary progressive MS (PPMS) patients ( n = 695). In a separate held-out test set of PPMS patients randomized to anti-CD20 antibodies or placebo ( n = 297), the average treatment effect was larger for the 50% (HR, 0.492; 95% CI, 0.266-0.912; p = 0.0218) and 30% (HR, 0.361; 95% CI, 0.165-0.79; p = 0.008) predicted to be most responsive, compared to 0.743 (95% CI, 0.482-1.15; p = 0.179) for the entire group. The same model could also identify responders to laquinimod in another held-out test set of PPMS patients ( n = 318). Finally, we show that using this model for predictive enrichment results in important increases in power.

2022-09-25

Nature Communications (publié)