Irina Rish

Ivan Anokhin

PhD - Université de Montréal

Co-supervisor :

Samira Ebrahimi Kahou

ivan.anokhin@mila.quebec

Rifat Arefin

PhD - Université de Montréal

arefinmr@mila.quebec

Arjun Ashok

PhD

Co-supervisor :

Alexandre Drouin

arjun.ashok@mila.quebec

Reza Bayat

Master's Research - Université de Montréal

Co-supervisor :

Pouya Bashivan

reza.bayat@mila.quebec

rishika.bhagwatkar@mila.quebec

Rishika Bhagwatkar

Master's Research - Université de Montréal

Collaborating researcher - Université de Montréal

connor.brennan@mila.quebec

Ethan Caballero

PhD - McGill University

Principal supervisor :

Blake Richards

ethan.caballero@mila.quebec

paolo.cudrano@mila.quebec

Paolo Cudrano

Collaborating researcher - Politecnico di Milano

Mohammad-Javad Darvishi Bayazi

Amin Darabi

PhD - Université de Montréal

amin.darabi@mila.quebec

PhD - Université de Montréal

mohammad-javad.darvishi-bayasi@mila.quebec

PhD - Université de Montréal

Co-supervisor :

PhD - Université de Montréal

faramarm@mila.quebec

Kshitij Gupta

Collaborating Alumni - Université de Montréal

Co-supervisor :

Sarath Chandar Anbil Parthipan

kshitij.gupta@mila.quebec

Nadhir Hassen

Research Intern - Université de Montréal

nadhir.hassen@mila.quebec

Prateek Humane

Collaborating researcher

prateek.humane@mila.quebec

PhD - Université de Montréal

Principal supervisor :

Nizar Islah

PhD - Université de Montréal

Principal supervisor :

Eilif Benjamin Muller

nizar.islah@mila.quebec

PhD - Université de Montréal

arnav-kumar.jain@mila.quebec

Collaborating researcher

ayush.kaushal@mila.quebec

Arian Khorasani

Master's Research - Université de Montréal

arian.khorasani@mila.quebec

Neeraj Kumar

Professional Master's - Université de Montréal

neeraj.kumar@mila.quebec

Gwen Legate

PhD - Concordia University

Principal supervisor :

gwendolyne.legate@mila.quebec

ekaterina.lobacheva@mila.quebec

Kate Lobacheva

Postdoctorate

Principal supervisor :

Nicolas Le Roux

Master's Research - Université de Montréal

sayed.mansouri-tehrani@mila.quebec

juan.mayor-torres@mila.quebec

Juan Manuel Mayor-Torres

Collaborating researcher

Independent visiting researcher

memariaa@mila.quebec

Andrei Mircea Romascanu

PhD - Université de Montréal

Master's Research - Université de Montréal

sparsha.mishra@mila.quebec

Diganta Misra

Master's Research - Université de Montréal

diganta.misra@mila.quebec

motahareh.pourrahimi@mila.quebec

Remus Mocanu

Master's Research - Université de Montréal

remus.mocanu@mila.quebec

Timothy Nest

PhD - Université de Montréal

Co-supervisor :

Eilif Benjamin Muller

timothy.nest@mila.quebec

Mohammad Pezeshki

Collaborating researcher

PhD - McGill University

Principal supervisor :

Pouya Bashivan

Maximilian Puelma Touzel

Collaborating researcher

Master's Research - Université de Montréal

Co-supervisor :

mahta.ramezanian@mila.quebec

Roland Riachi

Collaborating researcher - Université de Montréal

roland.riachi@mila.quebec

Matthew Riemer

PhD - Université de Montréal

matthew.riemer@mila.quebec

Alexis Roger

Master's Research - Université de Montréal

alexis.roger@mila.quebec

Zahra Sheikhbahaee

Postdoctorate - Université de Montréal

Principal supervisor :

zahra.sheikhbahaee@mila.quebec

Vaibhav Singh

PhD - Concordia University

Principal supervisor :

vaibhav.singh@mila.quebec

Gopeshh Subbaraj

PhD - Université de Montréal

gopeshh.subbaraj@mila.quebec

PhD - Université de Montréal

Co-supervisor :

benjamin.therien@mila.quebec

Pascal Tikeng Notsawo

PhD - Université de Montréal

Co-supervisor :

pascal.tikeng@mila.quebec

Master's Research - Université de Montréal

tejas.vaidhya@mila.quebec

Dave Whipps

Master's Research - Université de Montréal

whippsda@mila.quebec

andrew.williams@mila.quebec

Andrew Williams

PhD - Université de Montréal

daria.yasafova@mila.quebec

Daria Yasafova

Research Intern - Technical University of Munich

Omar Younis

Research Intern - Université de Montréal

omar.younis@mila.quebec

Publications

Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting

Kashif Rasul

Arjun Ashok

Andrew Robert Williams

Arian Khorasani

George Adamopoulos

Rishika Bhagwatkar

Marin Bilovs

Hena Ghonia

N. Hassen

Anderson Schneider

Sahil Garg

Alexandre Drouin

Nicolas Chapados

Yuriy Nevmyvaka

Over the past years, foundation models have caused a paradigm shift in machine learning due to their unprecedented capabilities for zero-sho… (see more)t and few-shot generalization. However, despite the success of foundation models in modalities such as natural language processing and computer vision, the development of foundation models for time series forecasting has lagged behind. We present Lag-Llama, a general-purpose foundation model for univariate probabilistic time series forecasting based on a decoder-only transformer architecture that uses lags as covariates. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities compared to a wide range of forecasting models on downstream datasets across domains. Moreover, when fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance, outperforming prior deep learning approaches, emerging as the best general-purpose model on average. Lag-Llama serves as a strong contender to the current state-of-art in time series forecasting and paves the way for future advancements in foundation models tailored to time series data.

2023-10-12

ArXiv (preprint)

arxiv.org

Comparison of Radiologists and Deep Learning for US Grading of Hepatic Steatosis.

Pedro Vianna

Sara-Ivana Calce

Pamela Boustros

Cassandra Larocque-Rigney

Laurent Patry-Beaudoin

Yi Hui Luo

Emre Aslan

John Marinos

Talal M. Alamri

Kim-Nhien Vu

Jessica Murphy-Lavallée

Jean-Sébastien Billiard

Emmanuel Montagnon

Hongliang Li

Samuel Kadoury

Bich Nguyen

Shanel Gauthier

Benjamin Thérien

Eugene Belilovsky … (see 4 more)

Guy Wolf

Michael Chassé

Guy Cloutier

An Tang

Background Screening for nonalcoholic fatty liver disease (NAFLD) is suboptimal due to the subjective interpretation of US images. Purpose T… (see more)o evaluate the agreement and diagnostic performance of radiologists and a deep learning model in grading hepatic steatosis in NAFLD at US, with biopsy as the reference standard. Materials and Methods This retrospective study included patients with NAFLD and control patients without hepatic steatosis who underwent abdominal US and contemporaneous liver biopsy from September 2010 to October 2019. Six readers visually graded steatosis on US images twice, 2 weeks apart. Reader agreement was assessed with use of κ statistics. Three deep learning techniques applied to B-mode US images were used to classify dichotomized steatosis grades. Classification performance of human radiologists and the deep learning model for dichotomized steatosis grades (S0, S1, S2, and S3) was assessed with area under the receiver operating characteristic curve (AUC) on a separate test set. Results The study included 199 patients (mean age, 53 years ± 13 [SD]; 101 men). On the test set (n = 52), radiologists had fair interreader agreement (0.34 [95% CI: 0.31, 0.37]) for classifying steatosis grades S0 versus S1 or higher, while AUCs were between 0.49 and 0.84 for radiologists and 0.85 (95% CI: 0.83, 0.87) for the deep learning model. For S0 or S1 versus S2 or S3, radiologists had fair interreader agreement (0.30 [95% CI: 0.27, 0.33]), while AUCs were between 0.57 and 0.76 for radiologists and 0.73 (95% CI: 0.71, 0.75) for the deep learning model. For S2 or lower versus S3, radiologists had fair interreader agreement (0.37 [95% CI: 0.33, 0.40]), while AUCs were between 0.52 and 0.81 for radiologists and 0.67 (95% CI: 0.64, 0.69) for the deep learning model. Conclusion Deep learning approaches applied to B-mode US images provided comparable performance with human readers for detection and grading of hepatic steatosis. Published under a CC BY 4.0 license. Supplemental material is available for this article. See also the editorial by Tuthill in this issue.

2023-10-01

Radiology (published)

Maximum State Entropy Exploration using Predecessor and Successor Representations

Arnav Kumar Jain

Lucas Lehnert

Glen Berseth

Animals have a developed ability to explore that aids them in important tasks such as locating food, exploring for shelter, and finding misp… (see more)laced items. These exploration skills necessarily track where they have been so that they can plan for finding items with relative efficiency. Contemporary exploration algorithms often learn a less efficient exploration strategy because they either condition only on the current state or simply rely on making random open-loop exploratory moves. In this work, we propose

WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series

Jean-Christophe Gagnon-Audet

Kartik Ahuja

Mohammad Javad Darvishi Bayazi

Pooneh Mousavi

2023-09-01

TMLR (accepted)

Beyond performance: the role of task demand, effort, and individual differences in ab initio pilots

Mohammad-Javad Darvishi-Bayazi

Andrew Law

Sergio Mejia Romero

Sion Jennings

Jocelyn Faubert

2023-08-28

Scientific Reports (published)

Neural efficiency in an aviation task with different levels of difficulty: Assessing different biometrics during a performance task

Mohammad Javad Darvishi Bayazi

Andrew Law

Sergio Mejia Romero

Sion Jennings

Jocelyn Faubert

2023-08-01

Journal of Vision (published)

Cognitive Models as Simulators: Using Cognitive Models to Tap into Implicit Human Feedback

Ardavan S. Nobandegani

Thomas Shultz

2023-06-20

ICML.cc/2023/Workshop/ILHF (published)

Continual Pre-Training of Large Language Models: How to (re)warm your model?

Kshitij Gupta

Benjamin Thérien

Adam Ibrahim

Mats Leon Richter

Quentin Gregory Anthony

Timothee LESORT

Large language models (LLMs) are routinely pre-trained on billions of tokens, only to restart the process over again once new data becomes a… (see more)vailable. A much cheaper and more efficient solution would be to enable the continual pre-training of these models, i.e. updating pre-trained models with new data instead of re-training them from scratch. However, the distribution shift induced by novel data typically results in degraded performance on past data. Taking a step towards efficient continual pre-training, in this work, we examine the effect of different warm-up strategies. Our hypothesis is that the learning rate must be re-increased to improve compute efficiency when training on a new dataset. We study the warmup phase of models pre-trained on the Pile (upstream data, 300B tokens) as we continue to pre-train on SlimPajama (downstream data, 297B tokens), following a linear warmup and cosine decay schedule. We conduct all experiments on the Pythia 410M language model architecture and evaluate performance through validation perplexity. We experiment with different pre-training checkpoints, various maximum learning rates, and various warmup lengths. Our results show that while rewarming models first increases the loss on upstream and downstream data, in the longer run it improves the downstream performance, outperforming models trained from scratch

2023-06-20

ICML.cc/2023/Workshop/ES-FoMO (poster)

Towards Out-of-Distribution Adversarial Robustness

Adam Ibrahim

Charles Guille-Escuret

Adversarial robustness continues to be a major challenge for deep learning. A core issue is that robustness to one type of attack often fail… (see more)s to transfer to other attacks. While prior work establishes a theoretical trade-off in robustness against different

2023-06-20

ICML.cc/2023/Workshop/AdvML-Frontiers (published)

Dialogue System with Missing Observation

Djallel Bouneffouf

Mayank Agarwal

Within the domain of dialogue, the ability to orchestrate multiple independently trained dialogue agents to create a unified system is of pa… (see more)rticular importance. Where we define orchestration as the task of selecting a subset of skills which most appropriately answer a user input using features extracted from both the user input and the individual skills. In this work, we study the task of online dialogue orchestration where the user feedback associated with the dialogue agent may not always be observed. In order to address the missing feedback setting, we propose to combine the attentive contextual bandit approach with an unsupervised learning mechanism such as clustering. By leveraging clustering to estimate missing reward, we are able to learn from each incoming event, even those with missing rewards. Promising empirical results are obtained on proprietary conversational datasets.

2023-06-04

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (published)

Estimating individual minimum calibration for deep-learning with predictive performance recovery: An example case of gait surface classification from wearable sensor gait data.

Guillaume Lam

P. Dixon

2023-06-01

Journal of Biomechanics (published)

Towards ethical multimodal systems

Alexis Roger

Esma Aimeur

2023-04-26

ArXiv (preprint)