Pablo Piantanida

Pierre Colombo

Implementing effective control mechanisms to ensure the proper functioning and security of deployed NLP models, from translation to chatbots… (voir plus), is essential. A key ingredient to ensure safe system behaviour is Out-Of-Distribution (OOD) detection, which aims to detect whether an input sample is statistically far from the training distribution. Although OOD detection is a widely covered topic in classification tasks, most methods rely on hidden features output by the encoder. In this work, we focus on leveraging soft-probabilities in a black-box framework, i.e. we can access the soft-predictions but not the internal states of the model. Our contributions include: (i) RAINPROOF a Relative informAItioN Projection OOD detection framework; and (ii) a more operational evaluation setting for OOD detection. Surprisingly, we find that OOD detection is not necessarily aligned with task-specific measures. The OOD detector may filter out samples well processed by the model and keep samples that are not, leading to weaker performance. Our results show that RAINPROOF provides OOD detection methods more aligned with task-specific performance metrics than traditional OOD detectors.

2023-12-01

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (publié)

Toward Stronger Textual Attack Detectors

Pierre Colombo

Marine Picot

Nathan Noiry

Guillaume Staerman

The landscape of available textual adversarial attacks keeps growing, posing severe threats and raising concerns regarding the deep NLP syst… (voir plus)em's integrity. However, the crucial problem of defending against malicious attacks has only drawn the attention of the NLP community. The latter is nonetheless instrumental in developing robust and trustworthy systems. This paper makes two important contributions in this line of search: (i) we introduce LAROUSSE, a new framework to detect textual adversarial attacks and (ii) we introduce STAKEOUT, a new benchmark composed of nine popular attack methods, three datasets, and two pre-trained models. LAROUSSE is ready-to-use in production as it is unsupervised, hyperparameter-free, and non-differentiable, protecting it against gradient-based methods. Our new benchmark STAKEOUT allows for a robust evaluation framework: we conduct extensive numerical experiments which demonstrate that LAROUSSE outperforms previous methods, and which allows to identify interesting factors of detection rate variations.

2023-12-01

Findings of the Association for Computational Linguistics: EMNLP 2023 (publié)

A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification

Pierre Colombo

Nathan Noiry

Guillaume Staerman

2023-10-21

ArXiv (prépublication)

Fundamental Limits of Membership Inference Attacks on Machine Learning Models

Eric Aubinais

Elisabeth Gassiat

Membership inference attacks (MIA) can reveal whether a particular data point was part of the training dataset, potentially exposing sensiti… (voir plus)ve information about individuals. This article provides theoretical guarantees by exploring the fundamental statistical limitations associated with MIAs on machine learning models. More precisely, we first derive the statistical quantity that governs the effectiveness and success of such attacks. We then deduce that in a very general regression setting with overfitting algorithms, attacks may have a high probability of success. Finally, we investigate several situations for which we provide bounds on this quantity of interest. Our results enable us to deduce the accuracy of potential attacks based on the number of samples and other structural parameters of learning models. In certain instances, these parameters can be directly estimated from the dataset.

2023-10-20

ArXiv (prépublication)

RainProof: An Umbrella to Shield Text Generator from Out-Of-Distribution Data

Maxime DARRIN

Pierre Colombo

2023-10-07

EMNLP/2023/Conference (accepté)

openreview.net

Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models

Pierre Colombo

Victor Pellegrain

Malik Boudiaf

Victor Storchan

Myriam Tami

Ismail Ben Ayed

C'eline Hudelot

Proprietary and closed APIs are becoming increasingly common to process natural language, and are impacting the practical applications of na… (voir plus)tural language processing, including few-shot classification. Few-shot classification involves training a model to perform a new classification task with a handful of labeled data. This paper presents three contributions. First, we introduce a scenario where the embedding of a pre-trained model is served through a gated API with compute-cost and data-privacy constraints. Second, we propose a transductive inference, a learning paradigm that has been overlooked by the NLP community. Transductive inference, unlike traditional inductive learning, leverages the statistics of unlabeled data. We also introduce a new parameter-free transductive regularizer based on the Fisher-Rao loss, which can be used on top of the gated API embeddings. This method fully utilizes unlabeled data, does not share any label with the third-party API provider and could serve as a baseline for future research. Third, we propose an improved experimental setting and compile a benchmark of eight datasets involving multiclass classification in four different languages, with up to 151 classes. We evaluate our methods using eight backbone models, along with an episodic evaluation over 1,000 episodes, which demonstrate the superiority of transductive inference over the standard inductive setting.

2023-10-07

EMNLP/2023/Conference (accepté)

openreview.net

Open-Set Likelihood Maximization for Few-Shot Learning

Malik Boudiaf

Etienne Bennequin

Myriam Tami

Antoine Toubhans

Celine Hudelot

Ismail Ben Ayed

We tackle the Few-Shot Open-Set Recognition (FSOSR) problem, i.e. classifying instances among a set of classes for which we only have a few … (voir plus)labeled samples, while simultaneously detecting instances that do not belong to any known class. We explore the popular transductive setting, which leverages the unlabelled query instances at inference. Motivated by the observation that existing transductive methods perform poorly in open-set scenarios, we propose a generalization of the maximum likelihood principle, in which latent scores down-weighing the influence of potential outliers are introduced alongside the usual parametric model. Our formulation embeds supervision constraints from the support set and additional penalties discouraging overconfident predictions on the query set. We proceed with a block-coordinate descent, with the latent scores and parametric model co-optimized alternately, thereby benefiting from each other. We call our resulting formulation Open-Set Likelihood Optimization (OSLO). OSLO is interpretable and fully modular; it can be applied on top of any pre-trained model seamlessly. Through extensive experiments, we show that our method surpasses existing inductive and transductive methods on both aspects of open-set recognition, namely inlier classification and outlier detection. Code is available at https://github.com/ebennequin/few-shot-open-set.

2023-06-17

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (publié)

A Functional Data Perspective and Baseline On Multi-Layer Out-of-Distribution Detection

Eduardo Dadalto Câmara Gomes

Pierre Colombo

Guillaume Staerman

Nathan Noiry