Publications

Implicitly Bayesian Prediction Rules in Deep Learning
Bruno Mlodozeniec
David M. Krueger
Richard E. Turner
The Bayesian approach leads to coherent updates of predictions under new data, which makes adhering to Bayesian principles appealing in deci… (see more)sion-making contexts. Traditionally, integrating Bayesian principles into models like deep neural networks involves setting priors on parameters and approximating posteriors. This is done despite the fact that, typically, priors on parameters reflect any prior beliefs only insofar as they dictate function space behaviour. In this paper, we rethink this approach and consider what properties characterise a prediction rule as being Bayesian. Algorithms meeting such criteria can be deemed implicitly Bayesian — they make the same predictions as some Bayesian model, without explicitly manifesting priors and posteriors. We argue this might be a more fruitful approach towards integrating Bayesian principles into deep learning. In this paper, we propose how to measure how close a general prediction rule is to being implicitly Bayesian, and empirically evaluate multiple prediction strategies using our approach. We also show theoretically that agents relying on non-implicitly Bayesian prediction rules can be easily exploited in adversarial betting settings.
2024-05-26
approximateinference.org/AABI/2024/Symposium_Archival_Track (accepted)
proceedings.mlr.press
proceedings.mlr.press
An Introduction to Vision-Language Modeling
Florian Bordes
Richard Yuanzhe Pang
Anurag Ajay
Alexander C. Li
Adrien Bardes
Suzanne Petryk
Oscar Mañas
Zhiqiu Lin
Anas Mahmoud
Bargav Jayaraman
Mark Ibrahim
Melissa Hall
Yunyang Xiong
Jonathan Lebensold
Candace Ross
Srihari Jayakumar
Chuan Guo
Diane Bouchacourt
Haider Al-Tahan
Karthik Padthe … (see 22 more)
Vasu Sharma
Huijuan Xu 0001
Hu Xu
Xiaoqing Ellen Tan
Megan Richards
Samuel Lavoie
Pietro Astolfi
Reyhane Askari Hemmat
Jun Chen
Kushal Tirumala
Rim Assouel
Mazda Moayeri
Arjang Talattof
Kamalika Chaudhuri
Zechun Liu
Xilun Chen
Quentin Garrido
Karen Ullrich
Aishwarya Agrawal
Kate Saenko
Asli Celikyilmaz
Vikas Chandra
2024-05-26
ArXiv (preprint)
doi.org
arxiv.org
Neuro-GPT: Towards A Foundation Model for EEG
Wenhui Cui
Woojae Jeong
Philipp Thölke
Takfarinas Medani
Karim Jerbi CoCo Lab
Anand A. Joshi
Richard M. Leahy
To handle the scarcity and heterogeneity of electroencephalography (EEG) data for Brain-Computer Interface (BCI) tasks, and to harness the p… (see more)ower of large publicly available data sets, we propose Neuro-GPT, a foundation model consisting of an EEG encoder and a GPT model. The foundation model is pre-trained on a large-scale data set using a self-supervised task that learns how to reconstruct masked EEG segments. We then fine-tune the model on a Motor Imagery Classification task to validate its performance in a low-data regime (9 subjects). Our experiments demonstrate that applying a foundation model can significantly improve classification performance compared to a model trained from scratch, which provides evidence for the generalizability of the foundation model and its ability to address challenges of data scarcity and heterogeneity in EEG. The code is publicly available at github.com/wenhui0206/NeuroGPT.
2024-05-26
2024 IEEE International Symposium on Biomedical Imaging (ISBI) (published)
doi.org
arxiv.org
Phenospectral similarity as an index of ecological integrity
Patrick Osei Darko
Étienne Laliberté
Margaret Kalacska
J. Pablo Arroyo-Mora
Andrew Gonzalez
Juan Zuloaga
In collaboration with the International Union for the Conservation of Nature (IUCN) Taskforce on Biodiversity and Protected Areas, countries… (see more) worldwide are working to develop a new systematic approach to inform the Key Biodiversity Areas (KBAs) initiative. The goal is to map KBAs from the national to global scales with a baseline international standard in support of biodiversity conservation efforts. According to the IUCN standard, one of the five criteria used to identify potential KBAs, is the Ecological Integrity (EI) of the ecosystem. Sites identified with respect to EI must have an intact ecological community and be characterized by minimal anthropogenic disturbance. In this study, a new EI metric, phenospectral similarity (PSpecM), has been developed and implemented in Google Earth Engine to identify potential forest stands of high EI from a large set of candidate stands. The implementation of PSpecM requires a network of known reference sites of high EI and target ecological units of the same land cover type for comparison to help identify potential sites of high EI. Here, we tested PSpecM on a ∼12,000 km2 study area in the Laurentian region, Quebec, Canada, using Sentinel-2 and PlanetScope (Dove) satellite imagery. Considering the phenological effect on reflectance, we found a 2,700 km2 spatial extent, equivalent to approximately 22% of the study area, commonly delineated as potential areas of high EI by both PlanetScope (Dove) and Sentinel-2. Without consideration of phenology, the total area delineated as potential areas of high EI increased to 5,505 km2, equivalent to around 45% of the study area. Our results show that PSpecM can be computed for rapid assessments of forest stands to identify potential areas of high EI on a large geographic scale and serve as an additional conservation tool that can be applied to the ongoing global and national identification of KBAs.
2024-05-26
Frontiers in Environmental Science (published)
doi.org
Subject-Based Domain Adaptation for Facial Expression Recognition
Muhammad Osama Zeeshan
Muhammad Haseeb Aslam
Soufiane Belharbi
Alessandro Lameiras Koerich
Marco Pedersoli
Simon Bacon
Eric Granger
Adapting a deep learning model to a specific target individual is a challenging facial expression recognition (FER) task that may be achieve… (see more)d using unsupervised domain adaptation (UDA) methods. Although several UDA methods have been proposed to adapt deep FER models across source and target data sets, multiple subject-specific source domains are needed to accurately represent the intra-and inter-person variability in subject-based adaption. This paper considers the setting where domains correspond to individuals, not entire datasets. Unlike UDA, multi-source domain adaptation (MSDA) methods can leverage multiple source datasets to improve the accuracy and robustness of the target model. However, previous methods for MSDA adapt image classification models across datasets and do not scale well to a more significant number of source domains. This paper introduces a new MSDA method for subject-based domain adaptation in FER. It efficiently leverages information from multiple source subjects (labeled source domain data) to adapt a deep FER model to a single target individual (unlabeled target domain data). During adaptation, our subject-based MSDA first computes a between-source discrepancy loss to mitigate the domain shift among data from several source subjects. Then, a new strategy is employed to generate augmented confident pseudo-labels for the target subject, allowing a reduction in the domain shift between source and target subjects. Experiments1 performed on the challenging BioVid heat and pain dataset with 87 subjects and the UNBC-McMaster shoulder pain dataset with 25 subjects show that our subject-based MSDA can outperform state-of-the-art methods yet scale well to multiple subject-based source domains.
2024-05-26
2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG) (published)
doi.org
arxiv.org
Reproducibility Study on Adversarial Attacks Against Robust Transformer Trackers
Fatemeh Nourilenjan Nokabadi
Jean-François Lalonde
Christian Gagné
New transformer networks have been integrated into object tracking pipelines and have demonstrated strong performance on the latest benchmar… (see more)ks. This paper focuses on understanding how transformer trackers behave under adversarial attacks and how different attacks perform on tracking datasets as their parameters change. We conducted a series of experiments to evaluate the effectiveness of existing adversarial attacks on object trackers with transformer and non-transformer backbones. We experimented on 7 different trackers, including 3 that are transformer-based, and 4 which leverage other architectures. These trackers are tested against 4 recent attack methods to assess their performance and robustness on VOT2022ST, UAV123 and GOT10k datasets. Our empirical study focuses on evaluating adversarial robustness of object trackers based on bounding box versus binary mask predictions, and attack methods at different levels of perturbations. Interestingly, our study found that altering the perturbation level may not significantly affect the overall object tracking results after the attack. Similarly, the sparsity and imperceptibility of the attack perturbations may remain stable against perturbation level shifts. By applying a specific attack on all transformer trackers, we show that new transformer trackers having a stronger cross-attention modeling achieve a greater adversarial robustness on tracking datasets, such as VOT2022ST and GOT10k. Our results also indicate the necessity for new attack methods to effectively tackle the latest types of transformer trackers. The codes necessary to reproduce this study are available at https://github.com/fatemehN/ReproducibilityStudy.
2024-05-25
TMLR (accepted)
doi.org
openreview.net
Towards a Generic Representation of Combinatorial Problems for Learning-Based Approaches
Léo Boisvert
Hélène Verhaeghe
Quentin Cappart
2024-05-24
Integration of Constraint Programming, Artificial Intelligence, and Operations Research (published)
doi.org
arxiv.org
Manifold Metric: A Loss Landscape Approach for Predicting Model Performance
Pranshu Malviya
Jerry Huang
Quentin Fournier
A. Chandar
Determining the optimal model for a given task often requires training multiple models from scratch, which becomes impractical as dataset an… (see more)d model sizes grow. A more efficient alternative is to expand smaller pre-trained models, but this approach is underutilized due to a limited understanding of its impact on the training dynamics. Existing methods for quantifying this impact have notable limitations, including computation cost. To address this, we introduce a new perspective based on the loss landscape, which has been shown to contain a manifold of linearly connected minima. Specifically, we propose a metric that estimates the size of this manifold to study the impact of model expansion. Our experiments reveal a strong correlation between performance gains and our manifold metric, enabling more informed model comparison and offering a first step toward a geometry-driven approach for reliable model expansion. Notably, our metric outperforms other baselines, even when different types of expansion with equivalent number of parameters are applied to a model.
2024-05-23
ArXiv (preprint)
arxiv.org
arxiv.org
Neural computations in prosopagnosia
Simon Faghel-Soubeyrand
Anne-Raphaelle Richoz
Delphine Waeber
Jessica Woodhams
Frédéric Gosselin
Roberto Caldara
Ian Charest
We aimed to identify neural computations underlying the loss of face identification ability by modelling the brain activity of brain-lesione… (see more)d patient PS, a well-documented case of acquired pure prosopagnosia. We collected a large dataset of high-density electrophysiological (EEG) recordings from PS and neurotypicals while they completed a one-back task on a stream of face, object, animal and scene images. We found reduced neural decoding of face identity around the N170 window in PS, and conjointly revealed normal non-face identification in this patient. We used Representational Similarity Analysis (RSA) to correlate human EEG representations with those of deep neural network (DNN) models of vision and caption-level semantics, offering a window into the neural computations at play in patient PS’s deficits. Brain representational dissimilarity matrices (RDMs) were computed for each participant at 4 ms steps using cross-validated classifiers. PS’s brain RDMs showed significant reliability across sessions, indicating meaningful measurements of brain representations with RSA even in the presence of significant lesions. Crucially, computational analyses were able to reveal PS’s representational deficits in high-level visual and semantic brain computations. Such multi-modal data-driven characterisations of prosopagnosia highlight the complex nature of processes contributing to face recognition in the human brain. Highlights We assess the neural computations in the prosopagnosic patient PS using EEG, RSA, and deep neural networks Neural dynamics of brain-lesioned PS are reliably captured using RSA Neural decoding shows normal evidence for non-face individuation in PS Neural decoding shows abnormal neural evidence for face individuation in PS PS shows impaired high-level visual and semantic neural computations
2024-05-23
Cerebral Cortex (New York, NY) (published)
doi.org
Predicting the Impact of Model Expansion through the Minima Manifold: A Loss Landscape Perspective
Pranshu Malviya
Jerry Huang
Quentin Fournier
A. Chandar
The optimal model for a given task is often challenging to determine, requiring training multiple models from scratch which becomes prohibit… (see more)ive as dataset and model sizes grow. A more efficient alternative is to reuse smaller pre-trained models by expanding them, however, this is not widely adopted as how this impacts training dynamics remains poorly understood. While prior works have introduced statistics to measure these effects, they remain flawed. To rectify this, we offer a new approach for understanding and quantifying the impact of expansion through the lens of the loss landscape, which has been shown to contain a manifold of linearly connected minima. Building on this new perspective, we propose a metric to study the impact of expansion by estimating the size of the manifold. Experimental results show a clear relationship between gains in performance and manifold size, enabling the comparison of candidate models and presenting a first step towards expanding models more reliably based on geometric properties of the loss landscape.
2024-05-23
ArXiv (preprint)
doi.org
arxiv.org
Assortment Optimization with Visibility Constraints
Théo Barré
Omar El Housni
Andrea Lodi
2024-05-21
Integer Programming and Combinatorial Optimization (published)
doi.org
arxiv.org
Attention as an RNN
Leo Feng
Frederick Tung
Hossein Hajimirsadeghi
Mohamed Osama Ahmed
Yoshua Bengio
Greg Mori
The advent of Transformers marked a significant breakthrough in sequence modelling, providing a highly performant architecture capable of le… (see more)veraging GPU parallelism. However, Transformers are computationally expensive at inference time, limiting their applications, particularly in low-resource settings (e.g., mobile and embedded devices). Addressing this, we (1) begin by showing that attention can be viewed as a special Recurrent Neural Network (RNN) with the ability to compute its \textit{many-to-one} RNN output efficiently. We then (2) show that popular attention-based models such as Transformers can be viewed as RNN variants. However, unlike traditional RNNs (e.g., LSTMs), these models cannot be updated efficiently with new tokens, an important property in sequence modelling. Tackling this, we (3) introduce a new efficient method of computing attention's \textit{many-to-many} RNN output based on the parallel prefix scan algorithm. Building on the new attention formulation, we (4) introduce \textbf{Aaren}, an attention-based module that can not only (i) be trained in parallel (like Transformers) but also (ii) be updated efficiently with new tokens, requiring only constant memory for inferences (like traditional RNNs). Empirically, we show Aarens achieve comparable performance to Transformers on
2024-05-21
ArXiv (preprint)
doi.org
arxiv.org