Publications

Cross-lingual Open-Retrieval Question Answering for African Languages
Odunayo Ogundepo
Tajuddeen Gwadabe
Clara Rivera
Jonathan Clark
Sebastian Ruder
David Adelani
Bonaventure Dossou
Abdou Diop
Claytone Sikasote
Gilles Hacheme
Happy Buzaaba
Ignatius Ezeani
Rooweither Mabuya
Salomey Osei
Chris Emezue
Albert Kahira
Shamsuddeen Muhammad
Akintunde Oladipo
Abraham Owodunni
Atnafu Tonja … (see 24 more)
Iyanuoluwa Shode
Akari Asai
Anuoluwapo Aremu
Ayodele Awokoya
Bernard Opoku
Chiamaka Chukwuneke
Christine Mwase
Clemencia Siro
Stephen Arthur
Tunde Ajayi
Verrah Otiende
Andre Rubungo
Boyd Sinkala
Daniel Ajisafe
Emeka Onwuegbuzia
Falalu Lawan
Ibrahim Ahmad
Jesujoba Alabi
Chinedu Mbonu
Mofetoluwa Adeyemi
Mofya Phiri
Orevaoghene Ahia
Ruqayya Iro
Sonia Adhiambo
2022-12-31
Findings of the Association for Computational Linguistics: EMNLP 2023 (published)
doi.org
openreview.net
Cutting Planes from the Branch-and-Bound Tree: Challenges and Opportunities
Claudio Contardo
Andrea Lodi
Andrea Tramontani
2022-12-31
INFORMS J. Comput. (published)
doi.org
DASVDD: Deep Autoencoding Support Vector Data Descriptor for Anomaly Detection
Hadi Hojjati
Narges Armanfard
Semi-supervised anomaly detection aims to detect anomalies from normal samples using a model that is trained on normal data. With recent adv… (see more)ancements in deep learning, researchers have designed efficient deep anomaly detection methods. Existing works commonly use neural networks to map the data into a more informative representation and then apply an anomaly detection algorithm. In this paper, we propose a method, DASVDD, that jointly learns the parameters of an autoencoder while minimizing the volume of an enclosing hyper-sphere on its latent representation. We propose an anomaly score which is a combination of autoencoder's reconstruction error and the distance from the center of the enclosing hypersphere in the latent representation. Minimizing this anomaly score aids us in learning the underlying distribution of the normal class during training. Including the reconstruction error in the anomaly score ensures that DASVDD does not suffer from the common hypersphere collapse issue since the DASVDD model does not converge to the trivial solution of mapping all inputs to a constant point in the latent representation. Experimental evaluations on several benchmark datasets show that the proposed method outperforms the commonly used state-of-the-art anomaly detection algorithms while maintaining robust performance across different anomaly classes.
2022-12-31
IEEE Transactions on Knowledge and Data Engineering (published)
doi.org
arxiv.org
Deep Networks as Paths on the Manifold of Neural Representations
Richard D Lange
Devin Kwok
Jordan Kyle Matelsky
Xinyue Wang
David Rolnick
Konrad Paul Kording
2022-12-31
TAG-ML (published)
proceedings.mlr.press
proceedings.mlr.press
Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses
Tal Golan
JohnMark Taylor
Heiko Herbert Schütt
Benjamin Peters
Rowan Paolo Sommers
Katja Seeliger
Adrien Doerig
Paul Linton
Talia Konkle
Marcel van Gerven
Konrad Kording
Blake Richards
Tim Christian Kietzmann
Grace W. Lindsay
Nikolaus Kriegeskorte

An ideal vision model accounts for behavior and neurophysiology in both naturalistic conditions and designed lab experiments. Unlike psyc… (see more)hological theories, artificial neural networks (ANNs) actually perform visual tasks and generate testable predictions for arbitrary inputs. These advantages enable ANNs to engage the entire spectrum of the evidence. Failures of particular models drive progress in a vibrant ANN research program of human vision.

2022-12-31
Behavioral and Brain Sciences (published)
doi.org
Design and Application of Adaptive Sparse Deep Echo State Network
Cuili Yang
Sheng Yang
Jian Tang
Bing Li
The prediction of appliances energy consumption in building belongs to time series forecasting problem, which can be solved by echo state ne… (see more)twork (ESN). However, due to the randomly initialized inputs and reservoir, some redundant or irrelevant components are inevitably generated in original ESN. To solve this problem, the adaptive sparse deep echo state network (ASDESN) is proposed, in which the information is processed layer by layer. Firstly, the principal component analysis (PCA) layer is inserted to penalize the redundant projection transmitted between sub-reservoirs. Secondly, the coordinate descent based adaptive sparse learning method is proposed to generate the sparse output weights. Particularly, the designed adaptive threshold strategy is able to enlarge the sparsity of output weights as network depth increases. Moreover, the echo state property (ESP) of ASDESN is given to ensure its applications. The experiment results in both simulated benchmark and real appliances energy datasets illustrate that the proposed ASDESN outperforms other ESNs with higher prediction accuracy and stability.
2022-12-31
IEEE Transactions on Consumer Electronics (published)
doi.org
Differential and Overlapping Effects between Exogenous and Endogenous Attention Shape Perceptual Facilitation during Visual Processing.
Mathieu Landry
Jason da Silva Castanheira
Karim Jerbi
Visuospatial attention is not a monolithic process and can be divided into different functional systems. In this framework, exogenous attent… (see more)ion reflects the involuntary orienting of attention resources following a salient event, whereas endogenous attention corresponds to voluntary orienting based on the goals and intentions of individuals. Previous work shows that these attention processes map onto distinct functional systems, yet evidence suggests that they are not fully independent. In the current work, we investigated the differential and overlapping effects of exogenous and endogenous attention on visual processing. We combined spatial cueing of visuospatial attention, EEG, and multivariate pattern analysis to examine where and when the effects of exogenous and endogenous attention were maximally different and maximally similar. Critically, multivariate pattern analysis provided new insights by examining whether classifiers trained to decode the cueing effect for one attention process (e.g., exogenous attention) can successfully decode the cueing effect for the other attention process (e.g., endogenous attention). These analyses uncovered differential and overlapping effects between exogenous and endogenous attention. Next, we combined principal component analyses, single-trial ERPs, and mediation analysis to determine whether these effects facilitate perception, as indexed by the behavioral spatial cueing effects of exogenous and endogenous attention. This approach revealed that three EEG components shape the cueing effects of exogenous and endogenous attention at various times after target onset. Altogether, our study provides a comprehensive account about how overlapping and differential processes of endogenous and exogenous relate to perceptual facilitation in the context of visuospatial attention.
2022-12-31
Journal of Cognitive Neuroscience (unknown)
doi.org
On Dynamic Program Decompositions of Static Risk Measures
Jia Lin Hau
Erick Delage
Mohammad Ghavamzadeh
Marek Petrik
Optimizing static risk-averse objectives in Markov decision processes is challenging because they do not readily admit dynamic programming d… (see more)ecompositions. Prior work has proposed to use a dynamic decomposition of risk measures that help to formulate dynamic programs on an augmented state space. This paper shows that several existing decompositions are inherently inexact, contradicting several claims in the literature. In particular, we give examples that show that popular decompositions for CVaR and EVaR risk measures are strict overestimates of the true risk values. However, an exact decomposition is possible for VaR, and we give a simple proof that illustrates the fundamental difference between VaR and CVaR dynamic programming properties.
2022-12-31
arXiv.org (preprint)
doi.org
Effective test generation using pre-trained Large Language Models and mutation testing
Arghavan Moradi Dakhel
Amin Nikanjam
Vahid Majdinasab
Foutse Khomh
Michel C. Desmarais
One of the critical phases in software development is software testing. Testing helps with identifying potential bugs and reducing maintenan… (see more)ce costs. The goal of automated test generation tools is to ease the development of tests by suggesting efficient bug-revealing tests. Recently, researchers have leveraged Large Language Models (LLMs) of code to generate unit tests. While the code coverage of generated tests was usually assessed, the literature has acknowledged that the coverage is weakly correlated with the efficiency of tests in bug detection. To improve over this limitation, in this paper, we introduce MuTAP for improving the effectiveness of test cases generated by LLMs in terms of revealing bugs by leveraging mutation testing. Our goal is achieved by augmenting prompts with surviving mutants, as those mutants highlight the limitations of test cases in detecting bugs. MuTAP is capable of generating effective test cases in the absence of natural language descriptions of the Program Under Test (PUTs). We employ different LLMs within MuTAP and evaluate their performance on different benchmarks. Our results show that our proposed method is able to detect up to 28% more faulty human-written code snippets. Among these, 17% remained undetected by both the current state-of-the-art fully automated test generation tool (i.e., Pynguin) and zero-shot/few-shot learning approaches on LLMs. Furthermore, MuTAP achieves a Mutation Score (MS) of 93.57% on synthetic buggy code, outperforming all other approaches in our evaluation. Our findings suggest that although LLMs can serve as a useful tool to generate test cases, they require specific post-processing steps to enhance the effectiveness of the generated test cases which may suffer from syntactic or functional errors and may be ineffective in detecting certain types of bugs and testing corner cases PUTs.
2022-12-31
arXiv (preprint)
doi.org
arxiv.org
An Empirical Investigation of the Role of Pre-training in Lifelong Learning
Sanket Vaibhav Mehta
Darshan Patil
Sarath Chandar
Emma Strubell
The lifelong learning paradigm in machine learning is an attractive alternative to the more prominent isolated learning scheme not only due … (see more)to its resemblance to biological learning but also its potential to reduce energy waste by obviating excessive model re-training. A key challenge to this paradigm is the phenomenon of catastrophic forgetting. With the increasing popularity and success of pre-trained models in machine learning, we pose the question: What role does pre-training play in lifelong learning, specifically with respect to catastrophic forgetting? We investigate existing methods in the context of large, pre-trained models and evaluate their performance on a variety of text and image classification tasks, including a large-scale study using a novel data set of 15 diverse NLP tasks. Across all settings, we observe that generic pre-training implicitly alleviates the effects of catastrophic forgetting when learning multiple tasks sequentially compared to randomly initialized models. We then further investigate why pre-training alleviates forgetting in this setting. We study this phenomenon by analyzing the loss landscape, finding that pre-trained weights appear to ease forgetting by leading to wider minima. Based on this insight, we propose jointly optimizing for current task loss and loss basin sharpness to explicitly encourage wider basins during sequential fine-tuning. We show that this optimization approach outperforms several state-of-the-art task-sequential continual learning algorithms across multiple settings, occasionally even without retaining a memory that scales in size with the number of tasks.
2022-12-31
Journal of Machine Learning Research (published)
doi.org
openreview.net
Explaining Graph Neural Networks Using Interpretable Local Surrogates
Farzaneh Heidari
Perouz Taslakian
Guillaume Rabusseau
2022-12-31
TAG-ML (published)
proceedings.mlr.press
proceedings.mlr.press
Exploring Self-Attention Mechanisms for Speech Separation
Cem Subakan
Mirco Ravanelli
Samuele Cornell
François Grondin
Mirko Bronzi
Transformers have enabled impressive improvements in deep learning. They often outperform recurrent and convolutional models in many tasks w… (see more)hile taking advantage of parallel processing. Recently, we proposed the SepFormer, which obtains state-of-the-art performance in speech separation with the WSJ0-2/3 Mix datasets. This paper studies in-depth Transformers for speech separation. In particular, we extend our previous findings on the SepFormer by providing results on more challenging noisy and noisy-reverberant datasets, such as LibriMix, WHAM!, and WHAMR!. Moreover, we extend our model to perform speech enhancement and provide experimental evidence on denoising and dereverberation tasks. Finally, we investigate, for the first time in speech separation, the use of efficient self-attention mechanisms such as Linformers, Lonformers, and ReFormers. We found that they reduce memory requirements significantly. For example, we show that the Reformer-based attention outperforms the popular Conv-TasNet model on the WSJ0-2Mix dataset while being faster at inference and comparable in terms of memory consumption.
2022-12-31
IEEE/ACM Transactions on Audio, Speech, and Language Processing (published)
doi.org
arxiv.org