Publications

Estimating causal effects with optimization-based methods: A review and empirical comparison
Martin Cousineau
Vedat Verter
Susan A. Murphy
Joelle Pineau
2023-01-01
European Journal of Operational Research (published)
doi.org
arxiv.org
Evaluating Dependencies in Fact Editing for Language Models: Specificity and Implication Awareness
Zichao Li
Ines Arous
Siva Reddy
Jackie Cheung
The potential of using a large language model (LLM) as a knowledge base (KB) has sparked significant interest. To maintain the knowledge acq… (see more)uired by LLMs, we need to ensure that the editing of learned facts respects internal logical constraints, which are known as dependency of knowledge. Existing work on editing LLMs has partially addressed the issue of dependency, when the editing of a fact should apply to its lexical variations without disrupting irrelevant ones. However, they neglect the dependency between a fact and its logical implications. We propose an evaluation protocol with an accompanying question-answering dataset, StandUp, that provides a comprehensive assessment of the editing process considering the above notions of dependency. Our protocol involves setting up a controlled environment in which we edit facts and monitor their impact on LLMs, along with their implications based on If-Then rules. Extensive experiments on StandUp show that existing knowledge editing methods are sensitive to the surface form of knowledge, and that they have limited performance in inferring the implications of edited facts.
2023-01-01
EMNLP (Findings) (published)
doi.org
openreview.net
Explaining Graph Neural Networks Using Interpretable Local Surrogates
Farzaneh Heidari
Perouz Taslakian
Guillaume Rabusseau
We propose an interpretable local surrogate (ILS) method for understanding the predictions of black-box graph models. Explainability methods… (see more) are commonly employed to gain insights into black-box models and, given the widespread adoption of GNNs in diverse applications, understanding the underlying reasoning behind their decision-making processes becomes crucial. Our ILS method approximates the behavior of a black-box graph model by fitting a simple surrogate model in the local neighborhood of a given input example. Leveraging the interpretability of the surrogate, ILS is able to identify the most relevant nodes contributing to a specific prediction. To efficiently identify these nodes, we utilize group sparse linear models as local surrogates. Through empirical evaluations on explainability benchmarks, our method consistently outperforms state-of-the-art graph explainability methods. This demonstrates the effectiveness of our approach in providing enhanced interpretability for GNN predictions.
2023-01-01
TAG-ML (published)
proceedings.mlr.press
Exploring Self-Attention Mechanisms for Speech Separation
Cem (Yusuf) Subakan
Mirco Ravanelli
Samuele Cornell
François Grondin
Mirko Bronzi
Transformers have enabled impressive improvements in deep learning. They often outperform recurrent and convolutional models in many tasks w… (see more)hile taking advantage of parallel processing. Recently, we proposed the SepFormer, which obtains state-of-the-art performance in speech separation with the WSJ0-2/3 Mix datasets. This paper studies in-depth Transformers for speech separation. In particular, we extend our previous findings on the SepFormer by providing results on more challenging noisy and noisy-reverberant datasets, such as LibriMix, WHAM!, and WHAMR!. Moreover, we extend our model to perform speech enhancement and provide experimental evidence on denoising and dereverberation tasks. Finally, we investigate, for the first time in speech separation, the use of efficient self-attention mechanisms such as Linformers, Lonformers, and ReFormers. We found that they reduce memory requirements significantly. For example, we show that the Reformer-based attention outperforms the popular Conv-TasNet model on the WSJ0-2Mix dataset while being faster at inference and comparable in terms of memory consumption.
2023-01-01
IEEE/ACM Transactions on Audio, Speech, and Language Processing (published)
doi.org
arxiv.org
Exploring trust development in families of children towards surgical and emergency care providers: A scoping review of the literature.
Olivia Serhan
Alexander Moise
Elena Guadagno
A. Issa
Dan Poenaru
2023-01-01
Journal of Pediatric Surgery (published)
doi.org
Expressiveness and Learnability: A Unifying View for Evaluating Self-Supervised Learning
Yuchen Lu
Zhen Liu
Aristide Baratin
Romain Laroche
Aaron Courville
Alessandro Sordoni
2023-01-01
Trans. Mach. Learn. Res. (published)
doi.org
openreview.net
Family risk communication preferences in pediatric surgery: A scoping review.
Arthega Selvarajan
Brandon Arulanandam
Elena Guadagno
Dan Poenaru
2023-01-01
Journal of Pediatric Surgery (published)
doi.org
Feature Likelihood Divergence: Evaluating the Generalization of Generative Models Using Samples
Marco Jiralerspong
Joey Bose
Ian Gemp
Chongli Qin
Yoram Bachrach
Gauthier Gidel
openreview.net
Feature Likelihood Score: Evaluating Generalization of Generative Models Using Samples
Marco Jiralerspong
Avishek Joey Bose
Gauthier Gidel
Deep generative models have demonstrated the ability to generate complex, high-dimensional, and photo-realistic data. However, a unified fr… (see more)amework for evaluating different generative modeling families remains a challenge. Indeed, likelihood-based metrics do not apply in many cases while pure sample-based metrics such as FID fail to capture known failure modes such as overfitting on training data. In this work, we introduce the Feature Likelihood Score (FLS), a parametric sample-based score that uses density estimation to quantitatively measure the quality/diversity of generated samples while taking into account overfitting. We empirically demonstrate the ability of FLS to identify specific overfitting problem cases, even when previously proposed metrics fail. We further perform an extensive experimental evaluation on various image datasets and model classes. Our results indicate that FLS matches intuitions of previous metrics, such as FID, while providing a more holistic evaluation of generative models that highlights models whose generalization abilities are under or overappreciated. Code for computing FLS is provided at https://github.com/marcojira/fls.
2023-01-01
arXiv.org (preprint)
doi.org
Filtering Pixel Latent Variables for Unmixing Volumetric Images
Catherine Bouchard
Vincent Boulanger
Flavie Lavoie-Cardinal
Christian Gagné
Measurements of different overlapping components require robust unmixing algorithms to convert the raw multi-dimensional measurements to use… (see more)ful unmixed images. Such algorithms perform reliable separation of the components when the raw signal is fully resolved and contains enough information to fit curves on the raw distributions. In experimental physics, measurements are often noisy, undersam-pled, or unresolved spatially or spectrally. We propose a novel method where bandpass filters are applied to the latent space of a multi-dimensional convolutional neural network to separate the overlapping signal components and extract each of their relative contributions. Simultaneously processing all dimensions with multi-dimensional convolution kernels empowers the network to combine the information from adjacent pixels and time-or spectral-bins, facilitating component separation in instances where individual pixels lack well-resolved information. We demonstrate the applicability of the method to real experimental physics problems using fluorescence lifetime microscopy and mode decomposition in optical fibers as test cases. The successful application of our approach to these two distinct experimental cases, characterized by different measured distributions, highlights the versatility of our approach in addressing a wide array of imaging tasks.
2023-01-01
arXiv.org (preprint)
doi.org
Filtering Pixel Latent Variables for Unmixing Volumetric Images
Catherine Bouchard
Vincent Boulanger
Flavie Lavoie-Cardinal
Christian Gagné
Measurements of different overlapping components require robust unmixing algorithms to convert the raw multi-dimensional measurements to use… (see more)ful unmixed images. Such algorithms perform reliable separation of the components when the raw signal is fully resolved and contains enough information to fit curves on the raw distributions. In experimental physics, measurements are often noisy, undersam-pled, or unresolved spatially or spectrally. We propose a novel method where bandpass filters are applied to the latent space of a multi-dimensional convolutional neural network to separate the overlapping signal components and extract each of their relative contributions. Simultaneously processing all dimensions with multi-dimensional convolution kernels empowers the network to combine the information from adjacent pixels and time-or spectral-bins, facilitating component separation in instances where individual pixels lack well-resolved information. We demonstrate the applicability of the method to real experimental physics problems using fluorescence lifetime microscopy and mode decomposition in optical fibers as test cases. The successful application of our approach to these two distinct experimental cases, characterized by different measured distributions, highlights the versatility of our approach in addressing a wide array of imaging tasks.
2023-01-01
arXiv.org (preprint)
doi.org
Findings of the 1st Shared Task on Multi-lingual Multi-task Information Retrieval at MRL 2023
Francesco Tinner
David Ifeoluwa Adelani
Chris Emezue
Mammad Hajili
Omer Goldman
Muhammad Farid Adilazuarda
Muhammad Dehan Al Kautsar
Aziza Mirsaidova
Müge Kural
Dylan Massey
Chiamaka Ijeoma Chukwuneke
CHINEDU EMMANUEL MBONU
Damilola Oluwaseun Oloyede
Kayode Olaleye
Jonathan Atala
Benjamin A. Ajibade
Saksham Bassi
Rahul Aralikatte
Najoung Kim
Duygu Ataman
Large language models (LLMs) excel in language understanding and generation, especially in English which has ample public benchmarks for var… (see more)ious natural language processing (NLP) tasks. Nevertheless, their reliability across different languages and domains remains uncertain. Our new shared task introduces a novel benchmark to assess the ability of multilingual LLMs to comprehend and produce language under sparse settings, particularly in scenarios with under-resourced languages, with an emphasis on the ability to capture logical, factual, or causal relationships within lengthy text contexts. The shared task consists of two sub-tasks crucial to information retrieval: Named Entity Recognition (NER) and Reading Comprehension (RC), in 7 data-scarce languages: Azerbaijani, Igbo, Indonesian, Swiss German, Turkish, Uzbek and Yorùbá, which previously lacked annotated resources in information retrieval tasks. Our evaluation of leading LLMs reveals that, despite their competitive performance, they still have notable weaknesses such as producing output in the non-target language or providing counterfactual information that cannot be inferred from the context. As more advanced models emerge, the benchmark will remain essential for supporting fairness and applicability in information retrieval systems.
2023-01-01
MRL (published)
doi.org