Publications

Cutting Planes from the Branch-and-Bound Tree: Challenges and Opportunities
Claudio Contardo
Andrea Lodi
Andrea Tramontani
2022-12-31
INFORMS J. Comput. (published)
doi.org
Deep Networks as Paths on the Manifold of Neural Representations
Richard D Lange
Devin Kwok
Jordan Kyle Matelsky
Xinyue Wang
David Rolnick
Konrad Paul Kording
2022-12-31
TAG-ML (published)
proceedings.mlr.press
proceedings.mlr.press
DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantization
Xinlin Li
Bang Liu
Rui Heng Yang
Vanessa Courville
Chao Xing
Vahid Partovi Nia
Efficiently deploying deep neural networks on low-resource edge devices is challenging due to their ever-increasing resource requirements. T… (see more)o address this issue, researchers have proposed multiplication-free neural networks, such as Power-of-Two quantization, or also known as Shift networks, which aim to reduce memory usage and simplify computation. However, existing low-bit Shift networks are not as accurate as their full-precision counterparts, typically suffering from limited weight range encoding schemes and quantization loss. In this paper, we propose the DenseShift network, which significantly improves the accuracy of Shift networks, achieving competitive performance to full-precision networks for vision and speech applications. In addition, we introduce a method to deploy an efficient DenseShift network using non-quantized floating-point activations, while obtaining 1.6X speed-up over existing methods. To achieve this, we demonstrate that zero-weight values in low-bit Shift networks do not contribute to model capacity and negatively impact inference computation. To address this issue, we propose a zero-free shifting mechanism that simplifies inference and increases model capacity. We further propose a sign-scale decomposition design to enhance training efficiency and a low-variance random initialization strategy to improve the model's transfer learning performance. Our extensive experiments on various computer vision and speech tasks demonstrate that DenseShift outperforms existing low-bit multiplication-free networks and achieves competitive performance compared to full-precision networks. Furthermore, our proposed approach exhibits strong transfer learning performance without a drop in accuracy. Our code was released on GitHub.
2022-12-31
IEEE/CVF Conference on Computer Vision and Pattern Recognition (published)
doi.org
arxiv.org
Differential and Overlapping Effects between Exogenous and Endogenous Attention Shape Perceptual Facilitation during Visual Processing.
Mathieu Landry
Jason da Silva Castanheira
Karim Jerbi
Visuospatial attention is not a monolithic process and can be divided into different functional systems. In this framework, exogenous attent… (see more)ion reflects the involuntary orienting of attention resources following a salient event, whereas endogenous attention corresponds to voluntary orienting based on the goals and intentions of individuals. Previous work shows that these attention processes map onto distinct functional systems, yet evidence suggests that they are not fully independent. In the current work, we investigated the differential and overlapping effects of exogenous and endogenous attention on visual processing. We combined spatial cueing of visuospatial attention, EEG, and multivariate pattern analysis to examine where and when the effects of exogenous and endogenous attention were maximally different and maximally similar. Critically, multivariate pattern analysis provided new insights by examining whether classifiers trained to decode the cueing effect for one attention process (e.g., exogenous attention) can successfully decode the cueing effect for the other attention process (e.g., endogenous attention). These analyses uncovered differential and overlapping effects between exogenous and endogenous attention. Next, we combined principal component analyses, single-trial ERPs, and mediation analysis to determine whether these effects facilitate perception, as indexed by the behavioral spatial cueing effects of exogenous and endogenous attention. This approach revealed that three EEG components shape the cueing effects of exogenous and endogenous attention at various times after target onset. Altogether, our study provides a comprehensive account about how overlapping and differential processes of endogenous and exogenous relate to perceptual facilitation in the context of visuospatial attention.
2022-12-31
Journal of Cognitive Neuroscience (unknown)
doi.org
On Dynamic Program Decompositions of Static Risk Measures
Jia Lin Hau
Erick Delage
Mohammad Ghavamzadeh
Marek Petrik
Optimizing static risk-averse objectives in Markov decision processes is challenging because they do not readily admit dynamic programming d… (see more)ecompositions. Prior work has proposed to use a dynamic decomposition of risk measures that help to formulate dynamic programs on an augmented state space. This paper shows that several existing decompositions are inherently inexact, contradicting several claims in the literature. In particular, we give examples that show that popular decompositions for CVaR and EVaR risk measures are strict overestimates of the true risk values. However, an exact decomposition is possible for VaR, and we give a simple proof that illustrates the fundamental difference between VaR and CVaR dynamic programming properties.
2022-12-31
arXiv.org (preprint)
doi.org
Energy Transition Pathways for Deep Decarbonization of the Greater Montreal Region: An Energy Optimization Framework
Sajad Aliakbari Sani
Azadeh Maroufmashat
Frédéric Babonneau
Olivier Bahn
Erick Delage
Alain Haurie
Normand Mousseau
Kathleen Vaillancourt
More than half of the world’s population live in cities, and by 2050, it is expected that this proportion will reach almost 68%. These den… (see more)sely populated cities consume more than 75% of the world’s primary energy and are responsible for the emission of around 70% of anthropogenic carbon. Providing sustainable energy for the growing demand in cities requires multifaceted planning approach. In this study, we modeled the energy system of the Greater Montreal region to evaluate the impact of different environmental mitigation policies on the energy system of this region over a long-term period (2020–2050). In doing so, we have used the open-source optimization-based model called the Energy–Technology–Environment Model (ETEM). The ETEM is a long-term bottom–up energy model that provides insight into the best options for cities to procure energy, and satisfies useful demands while reducing carbon dioxide (CO2) emissions. Results show that, under a deep decarbonization scenario, the transportation, commercial, and residential sectors will contribute to emission reduction by 6.9, 1.6, and 1 million ton CO2-eq in 2050, respectively, compared with their 2020 levels. This is mainly achieved by (i) replacing fossil fuel cars with electric-based vehicles in private and public transportation sectors; (ii) replacing fossil fuel furnaces with electric heat pumps to satisfy heating demand in buildings; and (iii) improving the efficiency of buildings by isolating walls and roofs.
2022-12-31
Energies (published)
doi.org
Explaining Graph Neural Networks Using Interpretable Local Surrogates
Farzaneh Heidari
Perouz Taslakian
Guillaume Rabusseau
2022-12-31
TAG-ML (published)
proceedings.mlr.press
proceedings.mlr.press
Exploring Self-Attention Mechanisms for Speech Separation
Cem Subakan
Mirco Ravanelli
Samuele Cornell
François Grondin
Mirko Bronzi
Transformers have enabled impressive improvements in deep learning. They often outperform recurrent and convolutional models in many tasks w… (see more)hile taking advantage of parallel processing. Recently, we proposed the SepFormer, which obtains state-of-the-art performance in speech separation with the WSJ0-2/3 Mix datasets. This paper studies in-depth Transformers for speech separation. In particular, we extend our previous findings on the SepFormer by providing results on more challenging noisy and noisy-reverberant datasets, such as LibriMix, WHAM!, and WHAMR!. Moreover, we extend our model to perform speech enhancement and provide experimental evidence on denoising and dereverberation tasks. Finally, we investigate, for the first time in speech separation, the use of efficient self-attention mechanisms such as Linformers, Lonformers, and ReFormers. We found that they reduce memory requirements significantly. For example, we show that the Reformer-based attention outperforms the popular Conv-TasNet model on the WSJ0-2Mix dataset while being faster at inference and comparable in terms of memory consumption.
2022-12-31
IEEE/ACM Transactions on Audio, Speech, and Language Processing (published)
doi.org
arxiv.org
Exploring trust development in families of children towards surgical and emergency care providers: A scoping review of the literature.
Olivia Serhan
Alexander Moise
Elena Guadagno
Amalia M. Issa
Dan Poenaru
2022-12-31
Journal of Pediatric Surgery (published)
doi.org
Family risk communication preferences in pediatric surgery: A scoping review.
Arthega Selvarajan
Brandon Arulanandam
Elena Guadagno
Dan Poenaru
2022-12-31
Journal of Pediatric Surgery (published)
doi.org
Filtering Pixel Latent Variables for Unmixing Volumetric Images
Catherine Bouchard
Vincent Boulanger
Flavie Lavoie-Cardinal
Christian Gagné
Measurements of different overlapping components require robust unmixing algorithms to convert the raw multi-dimensional measurements to use… (see more)ful unmixed images. Such algorithms perform reliable separation of the components when the raw signal is fully resolved and contains enough information to fit curves on the raw distributions. In experimental physics, measurements are often noisy, undersam-pled, or unresolved spatially or spectrally. We propose a novel method where bandpass filters are applied to the latent space of a multi-dimensional convolutional neural network to separate the overlapping signal components and extract each of their relative contributions. Simultaneously processing all dimensions with multi-dimensional convolution kernels empowers the network to combine the information from adjacent pixels and time-or spectral-bins, facilitating component separation in instances where individual pixels lack well-resolved information. We demonstrate the applicability of the method to real experimental physics problems using fluorescence lifetime microscopy and mode decomposition in optical fibers as test cases. The successful application of our approach to these two distinct experimental cases, characterized by different measured distributions, highlights the versatility of our approach in addressing a wide array of imaging tasks.
2022-12-31
arXiv.org (preprint)
doi.org
Findings of the 1st Shared Task on Multi-lingual Multi-task Information Retrieval at MRL 2023
Francesco Tinner
David Ifeoluwa Adelani
Chris Emezue
Mammad Hajili
Omer Goldman
Muhammad Farid Adilazuarda
Muhammad Dehan Al Kautsar
Aziza Mirsaidova
Müge Kural
Dylan Massey
Chiamaka Ijeoma Chukwuneke
CHINEDU EMMANUEL MBONU
Damilola Oluwaseun Oloyede
Kayode Olaleye
Jonathan Atala
Benjamin A. Ajibade
Saksham Bassi
Rahul Aralikatte
Najoung Kim
Duygu Ataman
Large language models (LLMs) excel in language understanding and generation, especially in English which has ample public benchmarks for var… (see more)ious natural language processing (NLP) tasks. Nevertheless, their reliability across different languages and domains remains uncertain. Our new shared task introduces a novel benchmark to assess the ability of multilingual LLMs to comprehend and produce language under sparse settings, particularly in scenarios with under-resourced languages, with an emphasis on the ability to capture logical, factual, or causal relationships within lengthy text contexts. The shared task consists of two sub-tasks crucial to information retrieval: Named Entity Recognition (NER) and Reading Comprehension (RC), in 7 data-scarce languages: Azerbaijani, Igbo, Indonesian, Swiss German, Turkish, Uzbek and Yorùbá, which previously lacked annotated resources in information retrieval tasks. Our evaluation of leading LLMs reveals that, despite their competitive performance, they still have notable weaknesses such as producing output in the non-target language or providing counterfactual information that cannot be inferred from the context. As more advanced models emerge, the benchmark will remain essential for supporting fairness and applicability in information retrieval systems.
2022-12-31
MRL (published)
doi.org