Publications

Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models
Zhong Zhang
Bang Liu
Junming Shao
2023-06-30
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (publié)
doi.org
arxiv.org
Genesis, modelling and methodological remedies to autism heterogeneity
Juliette Rabot
Eya‐mist Rødgaard
Ridha Joober
Guillaume Dumas
Danilo Bzdok
Boris C Bernhardt
Sébastien Jacquemont
Laurent Mottron
2023-06-30
Neuroscience & Biobehavioral Reviews (publié)
doi.org
Imaging of Neck Nodes in Head and Neck Cancers – a Comprehensive Update
K. Bhattacharya
A. Mahajan
R. Vaish
S. Rane
S. Shukla
A.K. D'Cruz
2023-06-30
Clinical Oncology (publié)
doi.org
MasakhaPOS: Part-of-Speech Tagging for Typologically Diverse African Languages
Cheikh M. Bamba Dione
David Ifeoluwa Adelani
Peter Nabende
Jesujoba O. Alabi
Thapelo Sindane
Happy Buzaaba
Shamsuddeen Hassan Muhammad
Chris Chinenye Emezue
Perez Ogayo
Anuoluwapo Aremu
Catherine Gitau
Derguene Mbaye
Jonathan Mukiibi
Blessing Sibanda
Bonaventure F. P. Dossou
Andiswa Bukula
Rooweither Mabuya
Allahsera Auguste Tapo
Edwin Munkoh-Buabeng
Victoire Memdjokam Koagne … (voir 24 de plus)
Fatoumata Ouoba Kabore
Amelia Taylor
Godson Kalipe
Tebogo Macucwa
Vukosi Marivate
Tajuddeen Gwadabe
Elvis Tchiaze Mboning
Ikechukwu Onyenwe
Gratien Atindogbe
Tolulope Anu Adelani
Idris Akinade
Olanrewaju Samuel
Marien Nahimana
Théogène Musabeyezu
Emile Niyomutabazi
Ester Chimhenga
Kudzai Gotosa
Patrick Mizha
Apelete Agbolo
Seydou Traore
Chinedu Uchechukwu
Aliyu Yusuf
Muhammad Abdullahi
Dietrich Klakow
In this paper, we present MasakhaPOS, the largest part-of-speech (POS) dataset for 20 typologically diverse African languages. We discuss th… (voir plus)e challenges in annotating POS for these languages using the UD (universal dependencies) guidelines. We conducted extensive POS baseline experiments using conditional random field and several multilingual pre-trained language models. We applied various cross-lingual transfer models trained with data available in UD. Evaluating on the MasakhaPOS dataset, we show that choosing the best transfer language(s) in both single-source and multi-source setups greatly improves the POS tagging performance of the target languages, in particular when combined with cross-lingual parameter-efficient fine-tuning methods. Crucially, transferring knowledge from a language that matches the language family and morphosyntactic properties seems more effective for POS tagging in unseen languages.
2023-06-30
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (publié)
doi.org
arxiv.org
Measuring Progress in Fine-grained Vision-and-Language Understanding
Emanuele Bugliarello
Laurent Sartran
Aishwarya Agrawal
Lisa Anne Hendricks
Aida Nematzadeh
While pretraining on large-scale image–text data from the Web has facilitated rapid progress on many vision-and-language (V&L) tasks, rece… (voir plus)nt work has demonstrated that pretrained models lack “fine-grained” understanding, such as the ability to recognise relationships, verbs, and numbers in images. This has resulted in an increased interest in the community to either develop new benchmarks or models for such capabilities. To better understand and quantify progress in this direction, we investigate four competitive V&L models on four fine-grained benchmarks. Through our analysis, we find that X-VLM (Zeng et al., 2022) consistently outperforms other baselines, and that modelling innovations can impact performance more than scaling Web data, which even degrades performance sometimes. Through a deeper investigation of X-VLM, we highlight the importance of both novel losses and rich data sources for learning fine-grained skills. Finally, we inspect training dynamics, and discover that for some tasks, performance peaks early in training or significantly fluctuates, never converging.
2023-06-30
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (publié)
doi.org
arxiv.org
Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation
Nuno M. Guerreiro
Pierre Colombo
Pablo Piantanida
André Martins
Neural machine translation (NMT) has become the de-facto standard in real-world machine translation applications. However, NMT models can un… (voir plus)predictably produce severely pathological translations, known as hallucinations, that seriously undermine user trust. It becomes thus crucial to implement effective preventive strategies to guarantee their proper functioning. In this paper, we address the problem of hallucination detection in NMT by following a simple intuition: as hallucinations are detached from the source content, they exhibit encoder-decoder attention patterns that are statistically different from those of good quality translations. We frame this problem with an optimal transport formulation and propose a fully unsupervised, plug-in detector that can be used with any attention-based NMT model. Experimental results show that our detector not only outperforms all previous model-based detectors, but is also competitive with detectors that employ external models trained on millions of samples for related tasks such as quality estimation and cross-lingual sentence similarity.
2023-06-30
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (publié)
doi.org
arxiv.org
Reinforcement Learning-Based Adaptive Feature Boosting for Smart Grid Intrusion Detection
Chengming Hu
Jun Yan
Xue Liu
Intrusion detection systems (IDSs) are crucial in the security monitoring for the smart grid with increasing machine-to-machine communicatio… (voir plus)ns and cyber threats thereafter. However, the multi-sourced, correlated, and heterogeneous smart grid data pose significant challenges to the accurate attack detection by IDSs. To improve the attack detection, this paper proposes Reinforcement Learning-based Adaptive Feature Boosting, which aims to leverage a series of AutoEncoders (AEs) to capture critical features from the multi-sourced smart grid data for the classification of normal, fault, and attack events. Multiple AEs are utilized to extract representative features from different feature sets that are automatically generated through a weighted feature sampling process; each AE-extracted feature set is then applied to build a Random Forest (RF) base classifier. In the feature sampling process, Deep Deterministic Policy Gradient (DDPG) is introduced to dynamically determine the feature sampling probability based on the classification accuracy. The critical features that improve the classification accuracy are assigned larger sampling probabilities and increasingly participate in the training of next AE. The presence of critical features is increased in the event classification over the multi-sourced smart grid data. Considering potential different alarms among base classifiers, an ensemble classifier is further built to distinguish normal, fault, and attack events. Our proposed approach is evaluated on the two realistic datasets collected from Hardware-In-the-Loop (HIL) and WUSTIL-IIOT-2021 security testbeds, respectively. The evaluation on the HIL security dataset shows that our proposed approach achieves the classification accuracy with 97.28%, an effective 5.5% increase over the vanilla Adaptive Feature Boosting. Moreover, the proposed approach not only accurately and stably selects critical features on the WUSTIL-IIOT-2021 dataset based on the significant difference of feature sampling probabilities between critical and uncritical features, i.e., the probabilities greater than 0.08 and less than 0.01, but also outperforms the other best-performing approaches with the increasing Matthew Correlation Coefficient (MCC) of 8.03%.
2023-06-30
IEEE Transactions on Smart Grid (publié)
doi.org
SkillQG: Learning to Generate Question for Reading Comprehension Assessment
Xiaoqiang Wang
Bang Liu
Siliang Tang
Lingfei Wu
2023-06-30
Findings of the Association for Computational Linguistics: ACL 2023 (publié)
doi.org
arxiv.org
The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources
Akshatha Arodi
Martin Pömsl
Kaheer Suleman
Adam Trischler
A.R. Olteanu
Jackie CK Cheung
Many state-of-the-art natural language understanding (NLU) models are based on pretrained neural language models. These models often make in… (voir plus)ferences using information from multiple sources. An important class of such inferences are those that require both background knowledge, presumably contained in a model’s pretrained parameters, and instance-specific information that is supplied at inference time. However, the integration and reasoning abilities of NLU models in the presence of multiple knowledge sources have been largely understudied. In this work, we propose a test suite of coreference resolution subtasks that require reasoning over multiple facts. These subtasks differ in terms of which knowledge sources contain the relevant facts. We also introduce subtasks where knowledge is present only at inference time using fictional knowledge. We evaluate state-of-the-art coreference resolution models on our dataset. Our results indicate that several models struggle to reason on-the-fly over knowledge observed both at pretrain time and at inference time. However, with task-specific training, a subset of models demonstrates the ability to integrate certain knowledge types from multiple sources. Still, even the best performing models seem to have difficulties with reliably integrating knowledge presented only at inference time.
2023-06-30
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (publié)
doi.org
arxiv.org
EuclidNets: An Alternative Operation for Efficient Inference of Deep Learning Models
Xinlin Li
Mariana Parazeres
Adam Oberman
Alireza Ghaffari
Masoud Asgharian
Vahid Nia
2023-06-29
SN Computer Science (publié)
doi.org
arxiv.org
Reference panel-guided super-resolution inference of Hi-C data
Yanlin Zhang
Mathieu Blanchette
Abstract Motivation Accurately assessing contacts between DNA fragments inside the nucleus with Hi-C experiment is crucial for understanding… (voir plus) the role of 3D genome organization in gene regulation. This challenging task is due in part to the high sequencing depth of Hi-C libraries required to support high-resolution analyses. Most existing Hi-C data are collected with limited sequencing coverage, leading to poor chromatin interaction frequency estimation. Current computational approaches to enhance Hi-C signals focus on the analysis of individual Hi-C datasets of interest, without taking advantage of the facts that (i) several hundred Hi-C contact maps are publicly available and (ii) the vast majority of local spatial organizations are conserved across multiple cell types. Results Here, we present RefHiC-SR, an attention-based deep learning framework that uses a reference panel of Hi-C datasets to facilitate the enhancement of Hi-C data resolution of a given study sample. We compare RefHiC-SR against tools that do not use reference samples and find that RefHiC-SR outperforms other programs across different cell types, and sequencing depths. It also enables high-accuracy mapping of structures such as loops and topologically associating domains. Availability and implementation https://github.com/BlanchetteLab/RefHiC.
2023-06-29
Bioinformatics (publié)
doi.org
Should We Feed the Trolls? Using Marketer-Generated Content to Explain Average Toxicity and Product Usage
Marcelo Vinhal Nepomuceno
Hooman Rahemi
Tolga Cenesizoglu
Laurent Charlin
2023-06-28
Journal of Interactive Marketing (publié)
doi.org