Doina Precup

busra-tugce.gurbuz@mila.quebec

Jesse Farebrother

Doctorat - McGill

Superviseur⋅e principal⋅e :

Marc Gendron-Bellemare

Doctorat - McGill

Superviseur⋅e principal⋅e :

Eilif B. Muller

Mohammad Sami Nur Islam Islam

Jonathan Hu

Maîtrise recherche - McGill

jonathan.hu@mila.quebec

Howard Huang

Doctorat - McGill

huanghow@mila.quebec

Maîtrise recherche - McGill

mohammad-sami-nur.islam@mila.quebec

Hangzhan Jin

Doctorat - Polytechnique

jinhangzhan@gmail.com

flemming.kondrup@mila.quebec

Flemming Kondrup

Postdoctorat - McGill

Jonathan Lebensold

Collaborateur·rice alumni - McGill

Collaborateur·rice alumni - McGill

jeremy.lucas@mila.quebec

Ray Luo

Doctorat - McGill

Superviseur⋅e principal⋅e :

Xujie Si

luo.ziyan@mila.quebec

Sicheng Lyu

Maîtrise recherche - McGill

sicheng.lyu@mila.quebec

G McCracken

Doctorat - McGill

mccrackg@mila.quebec

shahrad.mohammadzadeh@mila.quebec

Shahrad Mohammadzadeh

Maîtrise recherche - McGill

Superviseur⋅e principal⋅e :

Reihaneh Rabbany

padideh.nouri@mila.quebec

Gabriela Moisescu-Pareja

Collaborateur·rice de recherche - McGill

Co-superviseur⋅e :

Irina Rish

moisescg@mila.quebec

Padideh Nouri

Doctorat - UdeM

Doctorat - McGill

Co-superviseur⋅e :

Stagiaire de recherche - McGill

william.potter@mila.quebec

manoosh.samiei@mila.quebec

Nate Rahn

Doctorat - McGill

Superviseur⋅e principal⋅e :

Marc Gendron-Bellemare

nathan.rahn@mila.quebec

Mandana Samiei

Doctorat - McGill

Co-superviseur⋅e :

Blake Richards

samiemandana@gmail.com

Doctorat - McGill

Co-superviseur⋅e :

Paul Masset

Doctorat - McGill

Nishanth Anand Vemgal

Doctorat - McGill

Doctorat - McGill

Co-superviseur⋅e :

Samira Ebrahimi Kahou

Stagiaire de recherche - McGill

guangyuan.wang@mila.quebec

Zihan Wang

Doctorat - McGill

zihan.wang@mila.quebec

shuyuan.zhang@mila.quebec

Steve Wen

Maîtrise recherche - McGill

Co-superviseur⋅e :

Gregory Dudek

steve.wen@mila.quebec

Zijing Wu

Doctorat - McGill

Superviseur⋅e principal⋅e :

Paul Masset

zijing.wu@mila.quebec

Doctorat - McGill

Harry Zhao

Collaborateur·rice alumni - McGill

Co-superviseur⋅e :

Billets de blogue

Generic thumbnail for Mila Blog articles.

22 février 2024

Skipper : combiner l’abstraction spatiale et temporelle afin d’améliorer la généralisation

par

Mingde Harry Zhao

Safa Alver

Harm van Seijen

Romain Laroche

Doina Precup

Yoshua Bengio

Lire l'article

Publications

Predicting Future Disease Activity and Treatment Responders for Multiple Sclerosis Patients Using a Bag-of-Lesions Brain Representation

Andrew Doyle

Douglas Arnold

Tal Arbel

2017-09-09

International Conference on Medical Image Computing and Computer-Assisted Intervention (publié)

Horizontal and Vertical Self-Adaptive Cloud Controller with Reward Optimization for Resource Allocation

Jesús Alejandro Cárdenes Cabré

Ricardo Sanz

Over-booking or under-booking of computing resources leads to higher cost and performance degradation of web applications. To optimize the p… (voir plus)erformance of web applications, access to the resources has to be dynamically controlled ensuring maximum cost-performance ratio of the application while fulfilling requirements. To simplify the design of dynamic cloud controllers, we propose a horizontal and vertical scalability self-aware agent defined by a self-adaptive fuzzy logic with an oriented random optimizer based on reward and memory. The algorithm dynamically adjusts the membership functions and their relationship, maximizing the reward of the system while considering the cost related to the deployment of new resources. The evaluation of the controller under real cloud workload reveals the ability of the algorithm to maximize the performance of the web application based on the target parameters given by an operator.

2017-08-31

International Conference on Cloud and Autonomic Computing (publié)

World Knowledge for Reading Comprehension: Rare Entity Prediction with Hierarchical LSTMs Using External Descriptions

Teng Long

Emmanuel Bengio

Ryan Lowe

Jackie CK Cheung

Humans interpret texts with respect to some background information, or world knowledge, and we would like to develop automatic reading compr… (voir plus)ehension systems that can do the same. In this paper, we introduce a task and several models to drive progress towards this goal. In particular, we propose the task of rare entity prediction: given a web document with several entities removed, models are tasked with predicting the correct missing entities conditioned on the document context and the lexical resources. This task is challenging due to the diversity of language styles and the extremely large number of rare entities. We propose two recurrent neural network architectures which make use of external knowledge in the form of entity descriptions. Our experiments show that our hierarchical LSTM model performs significantly better at the rare entity prediction task than those that do not make use of external resources.

2017-08-31

Conference on Empirical Methods in Natural Language Processing (publié)

Approximate Value Iteration with Temporally Extended Actions (Extended Abstract)

Timothy A. Mann

Shie Mannor

The options framework provides a concrete way to implement and reason about temporally extended actions. Existing literature has demonstrate… (voir plus)d the value of planning with options empirically, but there is a lack of theoretical analysis formalizing when planning with options is more efficient than planning with primitive actions. We provide a general analysis of the convergence rate of a popular Approximate Value Iteration (AVI) algorithm called Fitted Value Iteration (FVI) with options. Our analysis reveals that longer duration options and a pessimistic estimate of the value function both lead to faster convergence. Furthermore, options can improve convergence even when they are suboptimal and sparsely distributed throughout the state space. Next we consider generating useful options for planning based on a subset of landmark states. This suggests a new algorithm, Landmarkbased AVI (LAVI), that represents the value function only at landmark states. We analyze OFVI and LAVI using the proposed landmark-based options and compare the two algorithms. Our theoretical and experimental results demonstrate that options can play an important role in AVI by decreasing approximation error and inducing fast convergence.

2017-08-18

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (publié)

Prediction of Extubation readiness in extremely preterm infants by the automated analysis of cardiorespiratory behavior: study protocol

Wissam Shalish

Lara J. Kanbar

Smita Rao

Carlos A. Robles-Rubio

Lajos Kovacs

Sanjay Chawla

Martin Keszler

Karen Brown

Robert E. Kearney

Guilherme M. Sant’Anna

BackgroundExtremely preterm infants (≤ 28 weeks gestation) commonly require endotracheal intubation and mechanical ventilation (MV) to ma… (voir plus)intain adequate oxygenation and gas exchange. Given that MV is independently associated with important adverse outcomes, efforts should be made to limit its duration. However, current methods for determining extubation readiness are inaccurate and a significant number of infants fail extubation and require reintubation, an intervention that may be associated with increased morbidities. A variety of objective measures have been proposed to better define the optimal time for extubation, but none have proven clinically useful. In a pilot study, investigators from this group have shown promising results from sophisticated, automated analyses of cardiorespiratory signals as a predictor of extubation readiness. The aim of this study is to develop an automated predictor of extubation readiness using a combination of clinical tools along with novel and automated measures of cardiorespiratory behavior, to assist clinicians in determining when extremely preterm infants are ready for extubation.MethodsIn this prospective, multicenter observational study, cardiorespiratory signals will be recorded from 250 eligible extremely preterm infants with birth weights ≤1250 g immediately prior to their first planned extubation. Automated signal analysis algorithms will compute a variety of metrics for each infant, and machine learning methods will then be used to find the optimal combination of these metrics together with clinical variables that provide the best overall prediction of extubation readiness. Using these results, investigators will develop an Automated system for Prediction of EXtubation (APEX) readiness that will integrate the software for data acquisition, signal analysis, and outcome prediction into a single application suitable for use by medical personnel in the neonatal intensive care unit. The performance of APEX will later be prospectively validated in 50 additional infants.DiscussionThe results of this research will provide the quantitative evidence needed to assist clinicians in determining when to extubate a preterm infant with the highest probability of success, and could produce significant improvements in extubation outcomes in this population.Trial registrationClinicaltrials.gov identifier: NCT01909947. Registered on July 17 2013.Trial sponsor: Canadian Institutes of Health Research (CIHR).

2017-07-16

BMC Pediatrics (publié)

APEX_SCOPE: A graphical user interface for visualization of multi-modal data in inter-disciplinary studies.

Lara J. Kanbar

Wissam Shalish

Karen A. Brown

Guilherme M. Sant'Anna

Robert E. Kearney

2017-07-10

2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (publié)

A Semi-Markov Chain Approach to Modeling Respiratory Patterns Prior to Extubation in Preterm Infants

Charles C. Onu

Lara J. Kanbar

Wissam Shalish

Karen A. Brown

Guilherme M. Sant'Anna

Robert E. Kearney

After birth, extremely preterm infants often require specialized respiratory management in the form of invasive mechanical ventilation (IMV)… (voir plus). Protracted IMV is associated with detrimental outcomes and morbidities. Premature extubation, on the other hand, would necessitate reintubation which is risky, technically challenging and could further lead to lung injury or disease. We present an approach to modeling respiratory patterns of infants who succeeded extubation and those who required reintubation which relies on Markov models. We compare the use of traditional Markov chains to semi-Markov models which emphasize cross-pattern transitions and timing information, and to multi-chain Markov models which can concisely represent non-stationarity in respiratory behavior over time. The models we developed expose specific, unique similarities as well as vital differences between the two populations.

2017-07-10

2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (publié)

arxiv.org

Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control

Policy gradient methods in reinforcement learning have become increasingly prevalent for state-of-the-art performance in continuous control … (voir plus)tasks. Novel methods typically benchmark against a few key algorithms such as deep deterministic policy gradients and trust region policy optimization. As such, it is important to present and use consistent baselines experiments. However, this can be difficult due to general variance in the algorithms, hyper-parameter tuning, and environment stochasticity. We investigate and discuss: the significance of hyper-parameters in policy gradients for continuous control, general variance in the algorithms, and reproducibility of reported results. We provide guidelines on reporting novel results as comparisons against baseline methods such that future researchers can make informed decisions when investigating novel methods.

2017-06-16

ICML.cc/2017/RML (poster)

openreview.net

Investigating Recurrence and Eligibility Traces in Deep Q-Networks

Jean Harb

Eligibility traces in reinforcement learning are used as a bias-variance trade-off and can often speed up training time by propagating knowl… (voir plus)edge back over time-steps in a single update. We investigate the use of eligibility traces in combination with recurrent networks in the Atari domain. We illustrate the benefits of both recurrent nets and eligibility traces in some Atari games, and highlight also the importance of the optimization used in the training.

2017-04-17

ArXiv (prépublication)

openreview.net

Multi-Timescale, Gradient Descent, Temporal Difference Learning with Linear Options

Peeyush T. Kumar

Deliberating on large or continuous state spaces have been long standing challenges in reinforcement learning. Temporal Abstraction have som… (voir plus)ewhat made this possible, but efficiently planing using temporal abstraction still remains an issue. Moreover using spatial abstractions to learn policies for various situations at once while using temporal abstraction models is an open problem. We propose here an efficient algorithm which is convergent under linear function approximation while planning using temporally abstract actions. We show how this algorithm can be used along with randomly generated option models over multiple time scales to plan agents which need to act real time. Using these randomly generated option models over multiple time scales are shown to reduce number of decision epochs required to solve the given task, hence effectively reducing the time needed for deliberation.

2017-03-18

ArXiv (prépublication)

arxiv.org

Learning-based interactive segmentation using the maximum mean cycle weight formalism

S. Nilufar

D. S. Wang

J. Girgis

C. G. Palii

D. Yang

A. Blais

M. Brand

T. J. Perkins

The maximum mean cycle weight (MMCW) segmentation framework is a graph-based alternative to approaches such as GraphCut or Markov Random Fie… (voir plus)lds. It offers time- and space-efficient computation and guaranteed optimality. However, unlike GraphCut or Markov Random Fields, MMCW does not seek to segment the entire image, but rather to find the single best object within the image, according to an objective function encoded by edge weights. Its focus on a single, best object makes MMCW attractive to interactive segmentation settings, where the user indicates which objects are to be segmented. However, a provably correct way of performing interactive segmentation using the MMCW framework has never been established. Further, the question of how to develop a good objective function based on user-provided information has never been addressed. Here, we propose a three-component objective function specifically designed for use with interactive MMCW segmentation. Two of those components, representing object boundary and object interior information, can be learned from a modest amount of user-labelled data, but in a way unique to the MMCW framework. The third component allows us to extend the MMCW framework to the situation of interactive segmentation. Specifically, we show that an appropriate weighted combination of the three components guarantees that the object produced by MMCW segmentation will enclose user-specified pixels that can be chosen interactively. The component weights can either be computed a priori based on image characteristics, or online via an adaptive reweighting scheme. We demonstrate the success of the approach on several microscope image segmentation problems.

2017-02-23

Medical Imaging (publié)

The Option-Critic Architecture

Pierre-Luc Bacon

Jean Harb

Temporal abstraction is key to scaling up learning and planning in reinforcement learning. While planning with temporally extended actions i… (voir plus)s well understood, creating such abstractions autonomously from data has remained challenging. We tackle this problem in the framework of options [Sutton, Precup & Singh, 1999; Precup, 2000]. We derive policy gradient theorems for options and propose a new option-critic architecture capable of learning both the internal policies and the termination conditions of options, in tandem with the policy over options, and without the need to provide any additional rewards or subgoals. Experimental results in both discrete and continuous environments showcase the flexibility and efficiency of the framework.

2017-02-12

Proceedings of the AAAI Conference on Artificial Intelligence (publié)