Publications

Were RNNs All We Needed?
Leo Feng
Frederick Tung
Mohamed Osama Ahmed
Yoshua Bengio
Hossein Hajimirsadeghi
The introduction of Transformers in 2017 reshaped the landscape of deep learning. Originally proposed for sequence modelling, Transformers h… (see more)ave since achieved widespread success across various domains. However, the scalability limitations of Transformers - particularly with respect to sequence length - have sparked renewed interest in novel recurrent models that are parallelizable during training, offer comparable performance, and scale more effectively. In this work, we revisit sequence modelling from a historical perspective, focusing on Recurrent Neural Networks (RNNs), which dominated the field for two decades before the rise of Transformers. Specifically, we examine LSTMs (1997) and GRUs (2014). We demonstrate that by simplifying these models, we can derive minimal versions (minLSTMs and minGRUs) that (1) use fewer parameters than their traditional counterparts, (2) are fully parallelizable during training, and (3) achieve surprisingly competitive performance on a range of tasks, rivalling recent models including Transformers.
2024-10-01
ArXiv (preprint)
doi.org
arxiv.org
Challenges for impact evaluation of WHO’s normative output
Catherine Régis
Gaëlle Foucault
Jean-Louis Denis
Pierre Larouche
Miriam Cohen
2024-09-30
Bulletin of the World Health Organization (published)
doi.org
Dynamics of scientific research on mirror neurons
H. Peyre
A. Yailian
G. Dumas
C. Gauld
2024-09-30
Neuropsychiatrie de l'Enfance et de l'Adolescence (published)
doi.org
Hybrid Simulator-Based Mechanism and Data-Driven for Multidemand Dioxin Emissions Intelligent Prediction in the MSWI Process
Heng Xia
Jian Tang
Wen Yu
JunFei Qiao
2024-09-30
IEEE transactions on industrial electronics (1982. Print) (published)
doi.org
Inferring electric vehicle charging patterns from smart meter data for impact studies
Feng Li
Élodie Campeau
Ilhan Kocar
Antoine Lesage-Landry
2024-09-30
Electric power systems research (published)
doi.org
Long-term outcomes of critically ill patients with hematological malignancies: what is the impact of the coronavirus disease 2019 pandemic? Author's reply
Laveena Munshi
Guillaume Dumas
Sangeeta Mehta
2024-09-30
Intensive Care Medicine (published)
doi.org
MAP: Model Merging with Amortized Pareto Front Using Limited Computation
Li Li
Tianyu Zhang
Zhiqi Bu
Suyuchen Wang
Huan He
Jie Fu
Yonghui Wu
Jiang Bian
Yong Chen
Yoshua Bengio
2024-09-30
NeurIPS.cc/2024/Workshop/Federated_Learning (oral)
openreview.net
openreview.net
Single-Shot Learning of Stable Dynamical Systems for Long-Horizon Manipulation Tasks
Alexandre St-Aubin
Amin Abyaneh
Hsiu-Chin Lin
Mastering complex sequential tasks continues to pose a significant challenge in robotics. While there has been progress in learning long-hor… (see more)izon manipulation tasks, most existing approaches lack rigorous mathematical guarantees for ensuring reliable and successful execution. In this paper, we extend previous work on learning long-horizon tasks and stable policies, focusing on improving task success rates while reducing the amount of training data needed. Our approach introduces a novel method that (1) segments long-horizon demonstrations into discrete steps defined by waypoints and subgoals, and (2) learns globally stable dynamical system policies to guide the robot to each subgoal, even in the face of sensory noise and random disturbances. We validate our approach through both simulation and real-world experiments, demonstrating effective transfer from simulation to physical robotic platforms. Code is available at https://github.com/Alestaubin/stable-imitation-policy-with-waypoints
2024-09-30
ArXiv (preprint)
doi.org
arxiv.org
softmax is not enough (for sharp out-of-distribution)
Petar Veličković
Christos Perivolaropoulos
Federico Barbero
Razvan Pascanu
A key property of reasoning systems is the ability to make sharp decisions on their input data. For contemporary AI systems, a key carrier o… (see more)f sharp behaviour is the softmax function, with its capability to perform differentiable query-key lookups. It is a common belief that the predictive power of networks leveraging softmax arises from "circuits" which sharply perform certain kinds of computations consistently across many diverse inputs. However, for these circuits to be robust, they would need to generalise well to arbitrary valid inputs. In this paper, we dispel this myth: even for tasks as simple as finding the maximum key, any learned circuitry must disperse as the number of items grows at test time. We attribute this to a fundamental limitation of the softmax function to robustly approximate sharp functions, prove this phenomenon theoretically, and propose adaptive temperature as an ad-hoc technique for improving the sharpness of softmax at inference time.
2024-09-30
ArXiv (preprint)
doi.org
arxiv.org
Spatial Action Unit Cues for Interpretable Deep Facial Expression Recognition
Soufiane Belharbi
Marco Pedersoli
Alessandro Lameiras Koerich
Simon Bacon
Eric Granger
Although state-of-the-art classifiers for facial expression recognition (FER) can achieve a high level of accuracy, they lack interpretabili… (see more)ty, an important feature for end-users. Experts typically associate spatial action units (AUs) from a codebook to facial regions for the visual interpretation of expressions. In this paper, the same expert steps are followed. A new learning strategy is proposed to explicitly incorporate AU cues into classifier training, allowing to train deep interpretable models. During training, this AU codebook is used, along with the input image expression label, and facial landmarks, to construct a AU heatmap that indicates the most discriminative image regions of interest w.r.t the facial expression. This valuable spatial cue is leveraged to train a deep interpretable classifier for FER. This is achieved by constraining the spatial layer features of a classifier to be correlated with AU heatmaps. Using a composite loss, the classifier is trained to correctly classify an image while yielding interpretable visual layer-wise attention correlated with AU maps, simulating the expert decision process. Our strategy only relies on image class expression for supervision, without additional manual annotations. Our new strategy is generic, and can be applied to any deep CNN- or transformer-based classifier without requiring any architectural change or significant additional training time. Our extensive evaluation on two public benchmarks RAF-DB, and AffectNet datasets shows that our proposed strategy can improve layer-wise interpretability without degrading classification performance. In addition, we explore a common type of interpretable classifiers that rely on class activation mapping (CAM) methods, and show that our approach can also improve CAM interpretability.
2024-09-30
ArXiv (preprint)
doi.org
arxiv.org
A Survey of Diversification Techniques in Search and Recommendation
Haolun Wu
Yansen Zhang
Chen Ma
Fuyuan Lyu
Bowei He
Fernando Diaz
Bhaskar Mitra
Xue Liu
Diversifying search results is an important research topic in retrieval systems in order to satisfy both the various interests of customers … (see more)and the equal market exposure of providers. There has been a growing attention on diversity-aware research during recent years, accompanied by a proliferation of literature on methods to promote diversity in search and recommendation. However, the diversity-aware studies in retrieval systems lack a systematic organization and are rather fragmented. In this survey, we are the first to propose a unified taxonomy for classifying the metrics and approaches of diversification in both search and recommendation, which are two of the most extensively researched fields of retrieval systems. We begin the survey with a brief discussion of why diversity is important in retrieval systems, followed by a summary of the various diversity concerns in search and recommendation, highlighting their relationship and differences. For the survey’s main body, we present a unified taxonomy of diversification metrics and approaches in retrieval systems, from both the search and recommendation perspectives. In the later part of the survey, we discuss the openness research questions of diversity-aware research in search and recommendation in an effort to inspire future innovations and encourage the implementation of diversity in real-world systems.
2024-09-30
IEEE Transactions on Knowledge and Data Engineering (published)
doi.org
arxiv.org
The Canadian VirusSeq Data Portal and Duotang: open resources for SARS-CoV-2 viral sequences and genomic epidemiology
Erin E. Gill
Baofeng Jia
Carmen Lia Murall
Raphaël Poujol
Muhammad Zohaib Anwar
Nithu Sara John
Justin Richardsson
Ashley Hobb
Abayomi S. Olabode
Alexandru Lepsa
Ana T. Duggan
Andrea D. Tyler
Arnaud N'Guessan
Atul Kachru
Brandon Chan
Catherine Yoshida
Christina K. Yung
David Bujold
Dusan Andric
Edmund Su … (see 46 more)
Emma J. Griffiths
Gary Van Domselaar
Gordon W. Jolly
Heather K. E. Ward
Henrich Feher
Jared Baker
Jared T. Simpson
Jaser Uddin
Jiannis Ragoussis
Jon Eubank
Jörg H. Fritz
José Héctor Gálvez
Karen Fang
Kim Cullion
Leonardo Rivera
Linda Xiang
Matthew A. Croxen
Mitchell Shiell
Natalie Prystajecky
Pierre-Olivier Quirion
Rosita Bajari
Samantha Rich
Samira Mubareka
Sandrine Moreira
Scott Cain
Steven G. Sutcliffe
Susanne A. Kraemer
Yelizar Alturmessov
Yann Joly
Marc Fiume
Terrance P. Snutch
Cindy Bell
Catalina López-Correa
Julie G. Hussin
Jeffrey B. Joy
Caroline Colijn
Paul M. K. Gordon
William W. L. Hsiao
Art F. Y. Poon
Natalie C. Knox
Mélanie Courtot
Lincoln Stein
Sarah P. Otto
Guillaume Bourque
B. Jesse Shapiro
Fiona S. L. Brinkman
The COVID-19 pandemic led to a large global effort to sequence SARS-CoV-2 genomes from patient samples to track viral evolution and inform t… (see more)he public health response. Millions of SARS-CoV-2 genome sequences have been deposited in global public repositories. The Canadian COVID-19 Genomics Network (CanCOGeN – VirusSeq), a consortium tasked with coordinating expanded sequencing of SARS-CoV-2 genomes across Canada early in the pandemic, created the Canadian VirusSeq Data Portal, with associated data pipelines and procedures, to support these efforts. The goal of VirusSeq was to allow open access to Canadian SARS-CoV-2 genomic sequences and enhanced, standardized contextual data that were unavailable in other repositories and that meet FAIR standards (Findable, Accessible, Interoperable and Reusable). In addition, the portal data submission pipeline contains data quality checking procedures and appropriate acknowledgement of data generators that encourages collaboration. From inception to execution, the portal was developed with a conscientious focus on strong data governance principles and practices. Extensive efforts ensured a commitment to Canadian privacy laws, data security standards, and organizational processes. This portal has been coupled with other resources, such as Viral AI, and was further leveraged by the Coronavirus Variants Rapid Response Network (CoVaRR-Net) to produce a suite of continually updated analytical tools and notebooks. Here we highlight this portal (https://virusseq-dataportal.ca/), including its contextual data not available elsewhere, and the Duotang (https://covarr-net.github.io/duotang/duotang.html), a web platform that presents key genomic epidemiology and modelling analyses on circulating and emerging SARS-CoV-2 variants in Canada. Duotang presents dynamic changes in variant composition of SARS-CoV-2 in Canada and by province, estimates variant growth, and displays complementary interactive visualizations, with a text overview of the current situation. The VirusSeq Data Portal and Duotang resources, alongside additional analyses and resources computed from the portal (COVID-MVP, CoVizu), are all open source and freely available. Together, they provide an updated picture of SARS-CoV-2 evolution to spur scientific discussions, inform public discourse, and support communication with and within public health authorities. They also serve as a framework for other jurisdictions interested in open, collaborative sequence data sharing and analyses.
2024-09-30
Microbial Genomics (published)
doi.org