Publications

Variational Nested Dropout
Yufei Cui
Yu Mao
Ziquan Liu
Qiao Li
Antoni B. Chan
Xue (Steve) Liu
Tei-Wei Kuo
Chun Jason Xue
Nested dropout is a variant of dropout operation that is able to order network parameters or features based on the pre-defined importance du… (see more)ring training. It has been explored for: I. Constructing nested nets Cui et al. 2020, Cui et al. 2021: the nested nets are neural networks whose architectures can be adjusted instantly during testing time, e.g., based on computational constraints. The nested dropout implicitly ranks the network parameters, generating a set of sub-networks such that any smaller sub-network forms the basis of a larger one. II. Learning ordered representation Rippel et al. 2014: the nested dropout applied to the latent representation of a generative model (e.g., auto-encoder) ranks the features, enforcing explicit order of the dense representation over dimensions. However, the dropout rate is fixed as a hyper-parameter during the whole training process. For nested nets, when network parameters are removed, the performance decays in a human-specified trajectory rather than in a trajectory learned from data. For generative models, the importance of features is specified as a constant vector, restraining the flexibility of representation learning. To address the problem, we focus on the probabilistic counterpart of the nested dropout. We propose a variational nested dropout (VND) operation that draws samples of multi-dimensional ordered masks at a low cost, providing useful gradients to the parameters of nested dropout. Based on this approach, we design a Bayesian nested neural network that learns the order knowledge of the parameter distributions. We further exploit the VND under different generative models for learning ordered latent distributions. In experiments, we show that the proposed approach outperforms the nested network in terms of accuracy, calibration, and out-of-domain detection in classification tasks. It also outperforms the related generative models on data generation tasks.
2023-08-01
IEEE Transactions on Pattern Analysis and Machine Intelligence (published)
doi.org
arxiv.org
Do visual mental imagery and exteroceptive perception rely on the same mechanisms?
Catherine Landry
Jasper JF van den Bosch
Ian Charest
Frédéric Gosselin
Vincent Taschereau-Dumouchel
2023-08-01
Journal of Vision (published)
doi.org
Multivariate analytical approaches for investigating brain-behavior relationships
E. Leighton Durham
Karam Ghanem
Andrew J. Stier
Carlos Cardenas-Iniguez
Gabrielle E. Reimann
Hee Jung Jeong
Randolph M. Dupont
Xiaoyu Dong
Tyler M. Moore
Marc G. Berman
Benjamin B. Lahey
Danilo Bzdok
Antonia N. Kaczkurkin
2023-07-31
Frontiers in Neuroscience (published)
doi.org
FASHION AND SUSTAINABILITY: A SYSTEMATIC LITERATURE REVIEW
Osmud Rahman
Dingtao Hu
Benjamin Fung
2023-07-30
Global Fashion Management Conference (published)
doi.org
GPS++: Reviving the Art of Message Passing for Molecular Property Prediction
Dominic Masters
Josef Dean
Kerstin Klaeser
Zhiyi Li
Samuel Maddrell-Mander
Adam Sanders
Hatem Helal
Deniz Beker
Andrew William Fitzgibbon
Shenyang Huang
Ladislav Rampášek
Dominique Beaini
2023-07-30
TMLR (accepted)
doi.org
openreview.net
Online Interior-point Methods for Time-varying Equality-constrained Optimization
Jean-Luc Lupien
Iman Shames
Antoine Lesage-Landry
2023-07-30
ArXiv (preprint)
doi.org
arxiv.org
Repurposing Density Functional Theory to Suit Deep Learning
Alexander Mathiasen
Hatem Helal
Paul Balanca
Kerstin Klaeser
Josef Dean
Carlo Luschi
Dominique Beaini
Andrew William Fitzgibbon
Dominic Masters
Density Functional Theory (DFT) accurately predicts the properties of molecules given their atom types and positions, and often serves as gr… (see more)ound truth for molecular property prediction tasks. Neural Networks (NN) are popular tools for such tasks and are trained on DFT datasets, with the aim to approximate DFT at a fraction of the computational cost. Research in other areas of machine learning has shown that generalisation performance of NNs tends to improve with increased dataset size, however, the computational cost of DFT limits the size of DFT datasets. We present PySCFIPU, a DFT library that allows us to iterate on both dataset generation and NN training. We create QM10X, a dataset with 100M conformers, in 13 hours, on which we subsequently train SchNet in 12 hours. We show that the predictions of SchNet improve solely by increasing training data without incorporating further inductive biases.
2023-07-28
ICML.cc/2023/Workshop/SynS_and_ML (published)
openreview.net
openreview.net
What if We Enrich day-ahead Solar Irradiance Time Series Forecasting with Spatio-Temporal Context?
Oussama Boussif
Ghait Boukachab
Dan Assouline
Stefano Massaroli
Tianle Yuan
Loubna Benabbou
Yoshua Bengio
The global integration of solar power into the electrical grid could have a crucial impact on climate change mitigation, yet poses a challen… (see more)ge due to solar irradiance variability. We present a deep learning architecture which uses spatio-temporal context from satellite data for highly accurate day-ahead time-series forecasting, in particular Global Horizontal Irradiance (GHI). We provide a multi-quantile variant which outputs a prediction interval for each time-step, serving as a measure of forecasting uncertainty. In addition, we suggest a testing scheme that separates easy and difficult scenarios, which appears useful to evaluate model performance in varying cloud conditions. Our approach exhibits robust performance in solar irradiance forecasting, including zero-shot generalization tests at unobserved solar stations, and holds great promise in promoting the effective use of solar power and the resulting reduction of CO
2023-07-28
ICML.cc/2023/Workshop/SynS_and_ML (published)
openreview.net
openreview.net
What if We Enrich day-ahead Solar Irradiance Time Series Forecasting with Spatio-Temporal Context?
Oussama Boussif
Ghait Boukachab
Dan Assouline
Stefano Massaroli
Tianle Yuan
Loubna Benabbou
Yoshua Bengio
2023-07-28
ICML.cc/2023/Workshop/SynS_and_ML (published)
doi.org
openreview.net
Resolution enhancement with a task-assisted GAN to guide optical nanoscopy image analysis and acquisition
Catherine Bouchard
Theresa Wiesner
Andréanne Deschênes
Anthony Bilodeau
Benoit Turcotte
Christian Gagné
Flavie Lavoie-Cardinal
2023-07-27
Nature Machine Intelligence (published)
doi.org
The Canadian Open Neuroscience Platform—An open science framework for the neuroscience community
Rachel J. Harding
Patrick Bermudez
Alexander Bernier
Michael Beauvais
Lune Bellec
Sean Hill
Bartha M. Knoppers
Agah Karakuzu
Paul Pavlidis
Jean-Baptiste Poline
Jane Roskams
Nikola Stikov
Jessica Stone
Stephen Strother
Conp Consortium
Alan C. Evans
The Canadian Open Neuroscience Platform (CONP) takes a multifaceted approach to enabling open neuroscience, aiming to make research, data, a… (see more)nd tools accessible to everyone, with the ultimate objective of accelerating discovery. Its core infrastructure is the CONP Portal, a repository with a decentralized design, where datasets and analysis tools across disparate platforms can be browsed, searched, accessed, and shared in accordance with FAIR principles. Another key piece of CONP infrastructure is NeuroLibre, a preprint server capable of creating and hosting executable and fully reproducible scientific publications that embed text, figures, and code. As part of its holistic approach, the CONP has also constructed frameworks and guidance for ethics and data governance, provided support and developed resources to help train the next generation of neuroscientists, and has fostered and grown an engaged community through outreach and communications. In this manuscript, we provide a high-level overview of this multipronged platform and its vision of lowering the barriers to the practice of open neuroscience and yielding the associated benefits for both individual researchers and the wider community.
2023-07-27
PLOS Computational Biology (published)
doi.org
A machine learning framework for neighbor generation in metaheuristic search
De-You Liu
Defeng Liu
Vincent Perreault
Alain Hertz
Andrea Lodi
This paper presents a methodology for integrating machine learning techniques into metaheuristics for solving combinatorial optimization pro… (see more)blems. Namely, we propose a general machine learning framework for neighbor generation in metaheuristic search. We first define an efficient neighborhood structure constructed by applying a transformation to a selected subset of variables from the current solution. Then, the key of the proposed methodology is to generate promising neighbors by selecting a proper subset of variables that contains a descent of the objective in the solution space. To learn a good variable selection strategy, we formulate the problem as a classification task that exploits structural information from the characteristics of the problem and from high-quality solutions. We validate our methodology on two metaheuristic applications: a Tabu Search scheme for solving a Wireless Network Optimization problem and a Large Neighborhood Search heuristic for solving Mixed-Integer Programs. The experimental results show that our approach is able to achieve a satisfactory trade-offs between the exploration of a larger solution space and the exploitation of high-quality solution regions on both applications.
2023-07-26
Frontiers in Applied Mathematics and Statistics (published)
doi.org
arxiv.org