Publications

Learning to Rewrite: Generalized LLM-Generated Text Detection

Wei Hao

Ran Li

Weiliang Zhao

Junfeng Yang

Chengzhi Mao

Large language models (LLMs) can be abused at scale to create non-factual content and spread disinformation. Detecting LLM-generated content… (see more) is essential to mitigate these risks, but current classifiers often fail to generalize in open-world contexts. Prior work shows that LLMs tend to rewrite LLM-generated content less frequently, which can be used for detection and naturally generalizes to unforeseen data. However, we find that the rewriting edit distance between human and LLM content can be indistinguishable across domains, leading to detection failures. We propose training an LLM to rewrite input text, producing minimal edits for LLM-generated content and more edits for human-written text, deriving a distinguishable and generalizable edit distance difference across different domains. Experiments on text from 21 independent domains and three popular LLMs (e.g., GPT-4o, Gemini, and Llama-3) show that our classifier outperforms the state-of-the-art zero-shot classifier by up to 20.6% on AUROC score and the rewriting classifier by 9.2% on F1 score. Our work suggests that LLM can effectively detect machine-generated text if they are trained properly.

2024-08-08

ArXiv (preprint)

doi.org

arxiv.org

Stochastic Wiring of Cell Types Enhances Fitness by Generating Phenotypic Variability

Divyansha Lachi

Ann Huang

Augustine N. Mavor-Parker

Arna Ghosh

Blake Richards

Anthony Zador

The development of neural connectivity is a crucial biological process that gives rise to diverse brain circuits and behaviors. Neural devel… (see more)opment is a stochastic process, but this stochasticity is often treated as a nuisance to overcome rather than as a functional advantage. Here we use a computational model, in which connection probabilities between discrete cell types are genetically specified, to investigate the benefits of stochasticity in the development of neural wiring. We show that this model can be viewed as a generalization of a powerful class of artificial neural networks—Bayesian neural networks—where each network parameter is a sample from a distribution. Our results reveal that stochasticity confers a greater benefit in large networks and variable environments, which may explain its role in organisms with larger brains. Surprisingly, we find that the average fitness over a population of agents is higher than a single agent defined by the average connection probability. Our model reveals how developmental stochasticity, by inducing a form of non-heritable phenotypic variability, can increase the probability that at least some individuals will survive in rapidly changing, unpredictable environments. Our results suggest how stochasticity may be an important feature rather than a bug in neural development.

2024-08-08

bioRxiv (preprint)

doi.org

Critical dynamics in spontaneous EEG predict anesthetic-induced loss of consciousness and perturbational complexity

Charlotte Maschke

Jordan O’Byrne

Michele Angelo Colombo

Melanie Boly

Olivia Gosseries

Steven Laureys

Mario Rosanova

Karim Jerbi

Stefanie Blain-Moraes

2024-08-05

Communications Biology (published)

doi.org

Learning Hybrid Interpretable Models: Theory, Taxonomy, and Methods

Julien Ferry

gabriel laberge

Ulrich Aivodji

A hybrid model involves the cooperation of an interpretable model and a complex black box. At inference, any input of the hybrid model is as… (see more)signed to either its interpretable or complex component based on a gating mechanism. The advantages of such models over classical ones are two-fold: 1) They grant users precise control over the level of transparency of the system and 2) They can potentially perform better than a standalone black box since redirecting some of the inputs to an interpretable model implicitly acts as regularization. Still, despite their high potential, hybrid models remain under-studied in the interpretability/explainability literature. In this paper, we remedy this fact by presenting a thorough investigation of such models from three perspectives: Theory, Taxonomy, and Methods. First, we explore the theory behind the generalization of hybrid models from the Probably-Approximately-Correct (PAC) perspective. A consequence of our PAC guarantee is the existence of a sweet spot for the optimal transparency of the system. When such a sweet spot is attained, a hybrid model can potentially perform better than a standalone black box. Secondly, we provide a general taxonomy for the different ways of training hybrid models: the Post-Black-Box and Pre-Black-Box paradigms. These approaches differ in the order in which the interpretable and complex components are trained. We show where the state-of-the-art hybrid models Hybrid-Rule-Set and Companion-Rule-List fall in this taxonomy. Thirdly, we implement the two paradigms in a single method: HybridCORELS, which extends the CORELS algorithm to hybrid modeling. By leveraging CORELS, HybridCORELS provides a certificate of optimality of its interpretable component and precise control over transparency. We finally show empirically that HybridCORELS is competitive with existing hybrid models, and performs just as well as a standalone black box (or even better) while being partly transparent.

2024-08-05

TMLR (accepted)

doi.org

openreview.net

The effect of gestational age on short- and long-term complications following primary esophageal atresia repair

Mathias Johansen

Samuel Wasserman

Dan Poenaru

Jean Martin Laberge

Sam J. Daniel

Thomas Engelhardt

2024-08-03

Brazilian Journal of Anesthesiology (published)

doi.org

Are self-explanations from Large Language Models faithful?

Andreas Madsen

Sarath Chandar

Siva Reddy

2024-08-01

Findings of the Association for Computational Linguistics ACL 2024 (published)

doi.org

arxiv.org

Investigating Failures to Generalize for Coreference Resolution Models

Ian Porada

Alexandra Olteanu

Kaheer Suleman

Adam Trischler

Jackie Cheung

Coreference resolution models are often evaluated on multiple datasets. Datasets vary, however, in how coreference is realized -- i.e., how … (see more)the theoretical concept of coreference is operationalized in the dataset -- due to factors such as the choice of corpora and annotation guidelines. We investigate the extent to which errors of current coreference resolution models are associated with existing differences in operationalization across datasets (OntoNotes, PreCo, and Winogrande). Specifically, we distinguish between and break down model performance into categories corresponding to several types of coreference, including coreferring generic mentions, compound modifiers, and copula predicates, among others. This break down helps us investigate how state-of-the-art models might vary in their ability to generalize across different coreference types. In our experiments, for example, models trained on OntoNotes perform poorly on generic mentions and copula predicates in PreCo. Our findings help calibrate expectations of current coreference resolution models; and, future work can explicitly account for those types of coreference that are empirically associated with poor generalization when developing models.

2024-08-01

Findings of the Association for Computational Linguistics ACL 2024 (published)

doi.org

arxiv.org

Knowledge Distillation in Federated Learning: A Practical Guide

Alessio Mora

Irene Tenison

Paolo Bellavista

Irina Rish

2024-08-01

Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence (published)

doi.org

A Look at Value-Based Decision-Time vs. Background Planning Methods Across Different Settings

Safa Alver

Doina Precup

In model-based reinforcement learning (RL), an agent can leverage a learned model to improve its way of behaving in different ways. Two of t… (see more)he prevalent ways to do this are through decision-time and background planning methods. In this study, we are interested in understanding how the value-based versions of these two planning methods will compare against each other across different settings. Towards this goal, we first consider the simplest instantiations of value-based decision-time and background planning methods and provide theoretical results on which one will perform better in the regular RL and transfer learning settings. Then, we consider the modern instantiations of them and provide hypotheses on which one will perform better in the same settings. Finally, we perform illustrative experiments to validate these theoretical results and hypotheses. Overall, our findings suggest that even though value-based versions of the two planning methods perform on par in their simplest instantiations, the modern instantiations of value-based decision-time planning methods can perform on par or better than the modern instantiations of value-based background planning methods in both the regular RL and transfer learning settings.

2024-08-01

EWRL/2024/Workshop (accepted)

openreview.net