Publications

Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Heads

Youcef Kemiche

Slim Essid

Self-supervised learning (SSL) leverages large datasets of unlabeled speech to reach impressive performance with reduced amounts of annotate… (voir plus)d data. The high number of proposed approaches fostered the emergence of comprehensive benchmarks that evaluate their performance on a set of downstream tasks exploring various aspects of the speech signal. However, while the number of considered tasks has been growing, most proposals rely upon a single downstream architecture that maps the frozen SSL representations to the task labels. This study examines how benchmarking results are affected by changes in the probing head architecture. Interestingly, we found that altering the downstream architecture structure leads to significant fluctuations in the performance ranking of the evaluated models. Against common practices in speech SSL benchmarking, we evaluate larger-capacity probing heads, showing their impact on performance, inference costs, generalization and multi-level feature exploitation.

2024-08-02

Computer Speech & Language (inconnu)

doi.org

arxiv.org

The effect of gestational age on short- and long-term complications following primary esophageal atresia repair

Mathias Johansen

Samuel Wasserman

Dan Poenaru

Jean Martin Laberge

Sam J. Daniel

Thomas Engelhardt

2024-08-02

Brazilian Journal of Anesthesiology (publié)

doi.org

Are self-explanations from Large Language Models faithful?

Andreas Madsen

A. Chandar

Siva Reddy

2024-07-31

Findings of the Association for Computational Linguistics ACL 2024 (publié)

doi.org

arxiv.org

Deep Learning Approach for Changepoint Detection: Penalty Parameter Optimization

Tung L. Nguyen

Toby Dylan Hocking

Changepoint detection, a technique for identifying significant shifts within data sequences, is crucial in various fields such as finance, g… (voir plus)enomics, medicine, etc. Dynamic programming changepoint detection algorithms are employed to identify the locations of changepoints within a sequence, which rely on a penalty parameter to regulate the number of changepoints. To estimate this penalty parameter, previous work uses simple models such as linear models or decision trees. This study introduces a novel deep learning method for predicting penalty parameters, leading to demonstrably improved changepoint detection accuracy on large benchmark supervised labeled datasets compared to previous methods.

2024-07-31

arXiv (publié)

doi.org

ECBD: Evidence-Centered Benchmark Design for NLP

Yu Lu Liu

Su Lin Blodgett

Jackie Chi

Jackie CK Cheung

Kit Cheung

Q. Vera Liao

A.R. Olteanu

Ziang Xiao

Benchmarking is seen as critical to assessing progress in NLP. However, creating a benchmark involves many design decisions (e.g., which dat… (voir plus)asets to include, which metrics to use) that often rely on tacit, untested assumptions about what the benchmark is intended to measure or is actually measuring. There is currently no principled way of analyzing these decisions and how they impact the validity of the benchmark's measurements. To address this gap, we draw on evidence-centered design in educational assessments and propose Evidence-Centered Benchmark Design (ECBD), a framework which formalizes the benchmark design process into five modules. ECBD specifies the role each module plays in helping practitioners collect evidence about capabilities of interest. Specifically, each module requires benchmark designers to describe, justify, and support benchmark design choices -- e.g., clearly specifying the capabilities the benchmark aims to measure or how evidence about those capabilities is collected from model responses. To demonstrate the use of ECBD, we conduct case studies with three benchmarks: BoolQ, SuperGLUE, and HELM. Our analysis reveals common trends in benchmark design and documentation that could threaten the validity of benchmarks' measurements.

2024-07-31

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (publié)

doi.org

arxiv.org

Functional Acceleration for Policy Mirror Descent

Veronica Chelu

Doina Precup

2024-07-31

EWRL/2024/Workshop (accepté)

doi.org

openreview.net

Generational Information Transfer with Neuroevolution on Control Tasks

Maximilien Le Clei

Stav Bar-Sheshet

Pierre Bellec

Lune P Bellec

2024-07-31

Proceedings of the Genetic and Evolutionary Computation Conference Companion (publié)

doi.org

Investigating Failures to Generalize for Coreference Resolution Models

Ian Porada

A.R. Olteanu

Kaheer Suleman

Adam Trischler

Jackie CK Cheung

Coreference resolution models are often evaluated on multiple datasets. Datasets vary, however, in how coreference is realized -- i.e., how … (voir plus)the theoretical concept of coreference is operationalized in the dataset -- due to factors such as the choice of corpora and annotation guidelines. We investigate the extent to which errors of current coreference resolution models are associated with existing differences in operationalization across datasets (OntoNotes, PreCo, and Winogrande). Specifically, we distinguish between and break down model performance into categories corresponding to several types of coreference, including coreferring generic mentions, compound modifiers, and copula predicates, among others. This break down helps us investigate how state-of-the-art models might vary in their ability to generalize across different coreference types. In our experiments, for example, models trained on OntoNotes perform poorly on generic mentions and copula predicates in PreCo. Our findings help calibrate expectations of current coreference resolution models; and, future work can explicitly account for those types of coreference that are empirically associated with poor generalization when developing models.

2024-07-31

Findings of the Association for Computational Linguistics ACL 2024 (publié)

doi.org

arxiv.org

Knowledge Distillation for Federated Learning: a Practical Guide

Alessio Mora

Irene Tenison

Paolo Bellavista

Irina Rish

Federated Learning (FL) enables the training of Deep Learning models without centrally collecting possibly sensitive raw data. This paves th… (voir plus)e way for stronger privacy guarantees when building predictive models. The most used algorithms for FL are parameter-averaging based schemes (e.g., Federated Averaging) that, however, have well known limits: (i) Clients must implement the same model architecture; (ii) Transmitting model weights and model updates implies high communication cost, which scales up with the number of model parameters; (iii) In presence of non-IID data distributions, parameter-averaging aggregation schemes perform poorly due to client model drifts. Federated adaptations of regular Knowledge Distillation (KD) can solve and/or mitigate the weaknesses of parameter-averaging FL algorithms while possibly introducing other trade-offs. In this article, we provide a review of KD-based algorithms tailored for specific FL issues.

2024-07-31

Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence (publié)

doi.org

arxiv.org

A Literature Review on Detecting, Verifying, and Mitigating Online Misinformation

Arezo Bodaghi

Ketra A. Schmitt

Pierre Watine

Benjamin C. M. Fung

Social media use has transformed communication and made social interaction more accessible. Public microblogs allow people to share and acce… (voir plus)ss news through existing and social-media-created social connections and access to public news sources. These benefits also create opportunities for the spread of false information. False information online can mislead people, decrease the benefits derived from social media, and reduce trust in genuine news. We divide false information into two categories: unintentional false information, also known as misinformation; and intentionally false information, also known as disinformation and fake news. Given the increasing prevalence of misinformation, it is imperative to address its dissemination on social media platforms. This survey focuses on six key aspects related to misinformation: 1) clarify the definition of misinformation to differentiate it from intentional forms of false information; 2) categorize proposed approaches to manage misinformation into three types: detection, verification, and mitigation; 3) review the platforms and languages for which these techniques have been proposed and tested; 4) describe the specific features that are considered in each category; 5) compare public datasets created to address misinformation and categorize into prelabeled content-only datasets and those including users and their connections; and 6) survey fact-checking websites that can be used to verify the accuracy of information. This survey offers a comprehensive and unprecedented review of misinformation, integrating various methodological approaches, datasets, and content-, user-, and network-based approaches, which will undoubtedly benefit future research in this field.

2024-07-31

IEEE Transactions on Computational Social Systems (publié)

doi.org

Motion In-Betweening via Deep <inline-formula><tex-math notation="LaTeX">$\Delta$</tex-math><alternatives><mml:math><mml:mi>Δ</mml:mi></mml:math><inline-graphic xlink:href="oreshkin-ieq1-3309107.gif"/></alternatives></inline-formula>-Interpolator

Boris Oreshkin

Antonios Valkanas

Félix Harvey

Louis-Simon Ménard

Florent Bocquelet

Mark J. Coates

We show that the task of synthesizing human motion conditioned on a set of key frames can be solved more accurately and effectively if a dee… (voir plus)p learning based interpolator operates in the delta mode using the spherical linear interpolator as a baseline. We empirically demonstrate the strength of our approach on publicly available datasets achieving state-of-the-art performance. We further generalize these results by showing that the

2024-07-31

IEEE Transactions on Visualization and Computer Graphics (publié)

doi.org

Neural differential equations for temperature control in buildings under demand response programs

Mathieu Le Cam

2024-07-31

Applied Energy (publié)

doi.org