Publications

Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error

Ofir Nachum

Shixiang Shane Gu

In this work, we study the use of the Bellman equation as a surrogate objective for value prediction accuracy. While the Bellman equation is… (see more) uniquely solved by the true value function over all state-action pairs, we find that the Bellman error (the difference between both sides of the equation) is a poor proxy for the accuracy of the value function. In particular, we show that (1) due to cancellations from both sides of the Bellman equation, the magnitude of the Bellman error is only weakly related to the distance to the true value function, even when considering all state-action pairs, and (2) in the finite data regime, the Bellman equation can be satisfied exactly by infinitely many suboptimal solutions. This means that the Bellman error can be minimized without improving the accuracy of the value function. We demonstrate these phenomena through a series of propositions, illustrative toy examples, and empirical analysis in standard benchmark domains.

2022-07-18

International Conference on Machine Learning (Accept for Short Presentation)

doi.org

proceedings.mlr.press

Near-Optimal Glimpse Sequences for Improved Hard Attention Neural Network Training

William Harvey

Michael Teng

Frank N. Wood

Hard visual attention is a promising approach to reduce the computational burden of modern computer vision methodologies. However, hard atte… (see more)ntion mechanisms can be difficult and slow to train, which is especially costly for applications like neural architecture search where multiple networks must be trained. We introduce a method to amortise the cost of training by generating an extra supervision signal for a subset of the training data. This supervision is in the form of sequences of ‘good’ locations to attend to for each image. We find that the best method to generate supervision sequences comes from framing hard attention for image classification as a Bayesian optimal experimental design (BOED) problem. From this perspective, the optimal locations to attend to are those which provide the greatest expected reduction in the entropy of the classification distribution. We introduce methodology from the BOED literature to approximate this optimal behaviour and generate ‘near-optimal’ supervision sequences. We then present a hard attention network training objective that makes use of these sequences and show that it allows faster training than prior work. We finally demonstrate the utility of faster hard attention training by incorporating supervision sequences in a neural architecture search, resulting in hard attention architectures which can outperform networks with access to the entire image.

2022-07-17

2022 International Joint Conference on Neural Networks (IJCNN) (published)

doi.org

arxiv.org

A Novel Neural Network-Based Malware Severity Classification System

Miles Q. Li

Benjamin C. M. Fung

2022-07-17

Communications in Computer and Information Science (published)

doi.org

On the Effectiveness of Interpretable Feedforward Neural Network

Miles Q. Li

Benjamin C. M. Fung

Adel Abusitta

Deep learning models have achieved state-of-the-art performance in many classification tasks. However, most of them cannot provide an explan… (see more)ation for their classification results. Machine learning models that are interpretable are usually linear or piecewise linear and yield inferior performance. Non-linear models achieve much better classification performance, but it is usually hard to explain their classification results. As a counter-example, an interpretable feedforward neural network (IFFNN) is proposed to achieve both high classification performance and interpretability for malware detection. If the IFFNN can perform well in a more flexible and general form for other classification tasks while providing meaningful explanations, it may be of great interest to the applied machine learning community. In this paper, we propose a way to generalize the interpretable feedforward neural network to multi-class classification scenarios and any type of feedforward neural networks, and evaluate its classification performance and interpretability on interpretable datasets. We conclude by finding that the generalized IFFNNs achieve comparable classification performance to their normal feedforward neural network counterparts and provide meaningful explanations. Thus, this kind of neural network architecture has great practical use.

2022-07-17

2022 International Joint Conference on Neural Networks (IJCNN) (published)

doi.org

arxiv.org

Unified gene expression signature of novel NPM1 exon 5 mutations in acute myeloid leukemia

Véronique Lisi

Ève Blanchard

Michael Vladovsky

Éric Audemard

Albert Ferghaly

Sébastien Lemieux

Josée Hébert

Guy Sauvageau

Vincent-Philippe Lavallee

Visual Abstract

2022-07-17

Blood Advances (published)

doi.org

Generative Models of Brain Dynamics

Mahta Ramezanian-Panahi

Germán Abrevaya

Jean-christophe Gagnon-audet

Vikram Voleti

Irina Rish

Guillaume Dumas

2022-07-14

Frontiers in Artificial Intelligence (published)

doi.org

Attention for Inference Compilation

William Harvey

Andreas Munk

Atilim Güneş Baydin

Alexander Bergholm

Frank Wood

We present a new approach to automatic amortized inference in universal probabilistic programs which improves performance compared to curren… (see more)t methods. Our approach is a variation of inference compilation (IC) which leverages deep neural networks to approximate a posterior distribution over latent variables in a probabilistic program. A challenge with existing IC network architectures is that they can fail to model long-range dependencies between latent variables. To address this, we introduce an attention mechanism that attends to the most salient variables previously sampled in the execution of a probabilistic program. We demonstrate that the addition of attention allows the proposal distributions to better match the true posterior, enhancing inference about latent variables in simulators.

2022-07-13

Proceedings of the 12th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (published)

doi.org

arxiv.org

Predicting Adverse Radiation Effects in Brain Tumors After Stereotactic Radiotherapy With Deep Learning and Handcrafted Radiomics

Simon A. Keek

Manon Beuque

Sergey Primakov

Henry C. Woodruff

Avishek Chatterjee

Janita E. van Timmeren

Martin Vallières

Lizza E. L. Hendriks

Johannes Kraft

Nicolaus Andratschke

Steve E. Braunstein

Olivier Morin

Philippe Lambin

Introduction There is a cumulative risk of 20–40% of developing brain metastases (BM) in solid cancers. Stereotactic radiotherapy (SRT) en… (see more)ables the application of high focal doses of radiation to a volume and is often used for BM treatment. However, SRT can cause adverse radiation effects (ARE), such as radiation necrosis, which sometimes cause irreversible damage to the brain. It is therefore of clinical interest to identify patients at a high risk of developing ARE. We hypothesized that models trained with radiomics features, deep learning (DL) features, and patient characteristics or their combination can predict ARE risk in patients with BM before SRT. Methods Gadolinium-enhanced T1-weighted MRIs and characteristics from patients treated with SRT for BM were collected for a training and testing cohort (N = 1,404) and a validation cohort (N = 237) from a separate institute. From each lesion in the training set, radiomics features were extracted and used to train an extreme gradient boosting (XGBoost) model. A DL model was trained on the same cohort to make a separate prediction and to extract the last layer of features. Different models using XGBoost were built using only radiomics features, DL features, and patient characteristics or a combination of them. Evaluation was performed using the area under the curve (AUC) of the receiver operating characteristic curve on the external dataset. Predictions for individual lesions and per patient developing ARE were investigated. Results The best-performing XGBoost model on a lesion level was trained on a combination of radiomics features and DL features (AUC of 0.71 and recall of 0.80). On a patient level, a combination of radiomics features, DL features, and patient characteristics obtained the best performance (AUC of 0.72 and recall of 0.84). The DL model achieved an AUC of 0.64 and recall of 0.85 per lesion and an AUC of 0.70 and recall of 0.60 per patient. Conclusion Machine learning models built on radiomics features and DL features extracted from BM combined with patient characteristics show potential to predict ARE at the patient and lesion levels. These models could be used in clinical decision making, informing patients on their risk of ARE and allowing physicians to opt for different therapies.

2022-07-12

Frontiers in Oncology (published)

doi.org

Interpretable Malware Classification based on Functional Analysis

Miles Q. Li

Benjamin C. M. Fung

2022-07-10

Proceedings of the 17th International Conference on Software Technologies (published)

doi.org

Transfer functions: learning about a lagged exposure-outcome association in time-series data

Hiroshi Mamiya

Alexandra M. Schmidt

Erica E. M. Moodie

David L. Buckeridge

Many population exposures in time-series analysis, including food marketing, exhibit a time-lagged association with population health outcom… (see more)es such as food purchasing. A common approach to measuring patterns of associations over different time lags relies on a finite-lag model, which requires correct specification of the maximum duration over which the lagged association extends. However, the maximum lag is frequently unknown due to the lack of substantive knowledge or the geographic variation of lag length. We describe a time-series analytical approach based on an infinite lag specification under a transfer function model that avoids the specification of an arbitrary maximum lag length. We demonstrate its application to estimate the lagged exposure-outcome association in food environmental research: display promotion of sugary beverages with lagged sales.

2022-07-10

International Journal of Public Health (published)

doi.org

An Introduction to Lifelong Supervised Learning

Shagun Sodhani

Mojtaba Farmazi

Sanket Vaibhav Mehta

Pranshu Malviya

Mohamed Abdelsalam

Janarthanan Janarthanan

A. Chandar

This primer is an attempt to provide a detailed summary of the different facets of lifelong learning. We start with Chapter 2 which provides… (see more) a high-level overview of lifelong learning systems. In this chapter, we discuss prominent scenarios in lifelong learning (Section 2.4), provide 8 Introduction a high-level organization of different lifelong learning approaches (Section 2.5), enumerate the desiderata for an ideal lifelong learning system (Section 2.6), discuss how lifelong learning is related to other learning paradigms (Section 2.7), describe common metrics used to evaluate lifelong learning systems (Section 2.8). This chapter is more useful for readers who are new to lifelong learning and want to get introduced to the field without focusing on specific approaches or benchmarks. The remaining chapters focus on specific aspects (either learning algorithms or benchmarks) and are more useful for readers who are looking for specific approaches or benchmarks. Chapter 3 focuses on regularization-based approaches that do not assume access to any data from previous tasks. Chapter 4 discusses memory-based approaches that typically use a replay buffer or an episodic memory to save subset of data across different tasks. Chapter 5 focuses on different architecture families (and their instantiations) that have been proposed for training lifelong learning systems. Following these different classes of learning algorithms, we discuss the commonly used evaluation benchmarks and metrics for lifelong learning (Chapter 6) and wrap up with a discussion of future challenges and important research directions in Chapter 7.

2022-07-09

ArXiv (preprint)

doi.org

arxiv.org

Partial Disentanglement via Mechanism Sparsity

Sébastien Lachapelle

Simon Lacoste-Julien

2022-07-08

auai.org/UAI/2022/Workshop/CRL (oral)

doi.org

openreview.net