Publications

Unveiling the Flaws: A Critical Analysis of Initialization Effect on Time Series Anomaly Detection

Alex Koran

Hadi Hojjati

Narges Armanfard

Deep learning for time-series anomaly detection (TSAD) has gained significant attention over the past decade. Despite the reported improveme… (see more)nts in several papers, the practical application of these models remains limited. Recent studies have cast doubt on these models, attributing their results to flawed evaluation techniques. However, the impact of initialization has largely been overlooked. This paper provides a critical analysis of the initialization effects on TSAD model performance. Our extensive experiments reveal that TSAD models are highly sensitive to hyperparameters such as window size, seed number, and normalization. This sensitivity often leads to significant variability in performance, which can be exploited to artificially inflate the reported efficacy of these models. We demonstrate that even minor changes in initialization parameters can result in performance variations that overshadow the claimed improvements from novel model architectures. Our findings highlight the need for rigorous evaluation protocols and transparent reporting of preprocessing steps to ensure the reliability and fairness of anomaly detection methods. This paper calls for a more cautious interpretation of TSAD advancements and encourages the development of more robust and transparent evaluation practices to advance the field and its practical applications.

2024-08-13

ArXiv (preprint)

Can a Bayesian Oracle Prevent Harm from an Agent?

Yoshua Bengio

Michael K. Cohen

Nikolay Malkin

Matt MacDermott

Damiano Fornasiere

Pietro Greiner

Younesse Kaddar

Is there a way to design powerful AI systems based on machine learning methods that would satisfy probabilistic safety guarantees? With the … (see more)long-term goal of obtaining a probabilistic guarantee that would apply in every context, we consider estimating a context-dependent bound on the probability of violating a given safety specification. Such a risk evaluation would need to be performed at run-time to provide a guardrail against dangerous actions of an AI. Noting that different plausible hypotheses about the world could produce very different outcomes, and because we do not know which one is right, we derive bounds on the safety violation probability predicted under the true but unknown hypothesis. Such bounds could be used to reject potentially dangerous actions. Our main results involve searching for cautious but plausible hypotheses, obtained by a maximization that involves Bayesian posteriors over hypotheses. We consider two forms of this result, in the iid case and in the non-iid case, and conclude with open problems towards turning such theoretical results into practical AI guardrails.

2024-08-09

ArXiv (preprint)

Revisiting Feature Prediction for Learning Visual Representations from Video

Adrien Bardes

Quentin Garrido

Jean Ponce

Xinlei Chen

Michael Rabbat

Yann LeCun

Mahmoud Assran

Nicolas Ballas

2024-08-09

TMLR (accepted)

Cardinality Minimization, Constraints, and Regularization: A Survey

Andreas M. Tillmann

Daniel Bienstock

Andrea Lodi

Alexandra Schwartz

We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unif… (see more)ied viewpoint on the general problem classes and models, and give concrete examples from diverse application fields such as signal and image processing, portfolio selection, or machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically draws on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., shed light on structural properties of a model or its solutions, or lead to the development of efficient heuristics; we also provide some illustrative examples.

2024-08-08

SIAM Review (published)

Diminished social memory and hippocampal correlates of social interactions in chronic social defeat stress susceptibility

Amanda Larosa

Tian Rui Zhang

Alice S. Wong

Y. H. Fung Cyrus

Xiong Ling Yun (Jenny) Long

Benjamin Fung

Tak Pan Wong

2024-08-08

bioRxiv (preprint)

Learning to Rewrite: Generalized LLM-Generated Text Detection

Wei Hao

Ran Li

Weiliang Zhao

Junfeng Yang

Chengzhi Mao

Large language models (LLMs) can be abused at scale to create non-factual content and spread disinformation. Detecting LLM-generated content… (see more) is essential to mitigate these risks, but current classifiers often fail to generalize in open-world contexts. Prior work shows that LLMs tend to rewrite LLM-generated content less frequently, which can be used for detection and naturally generalizes to unforeseen data. However, we find that the rewriting edit distance between human and LLM content can be indistinguishable across domains, leading to detection failures. We propose training an LLM to rewrite input text, producing minimal edits for LLM-generated content and more edits for human-written text, deriving a distinguishable and generalizable edit distance difference across different domains. Experiments on text from 21 independent domains and three popular LLMs (e.g., GPT-4o, Gemini, and Llama-3) show that our classifier outperforms the state-of-the-art zero-shot classifier by up to 20.6% on AUROC score and the rewriting classifier by 9.2% on F1 score. Our work suggests that LLM can effectively detect machine-generated text if they are trained properly.

2024-08-08

ArXiv (preprint)

Stochastic Wiring of Cell Types Enhances Fitness by Generating Phenotypic Variability

Divyansha Lachi

Ann Huang

Augustine N. Mavor-Parker

Arna Ghosh

Blake Richards

Anthony Zador

The development of neural connectivity is a crucial biological process that gives rise to diverse brain circuits and behaviors. Neural devel… (see more)opment is a stochastic process, but this stochasticity is often treated as a nuisance to overcome rather than as a functional advantage. Here we use a computational model, in which connection probabilities between discrete cell types are genetically specified, to investigate the benefits of stochasticity in the development of neural wiring. We show that this model can be viewed as a generalization of a powerful class of artificial neural networks—Bayesian neural networks—where each network parameter is a sample from a distribution. Our results reveal that stochasticity confers a greater benefit in large networks and variable environments, which may explain its role in organisms with larger brains. Surprisingly, we find that the average fitness over a population of agents is higher than a single agent defined by the average connection probability. Our model reveals how developmental stochasticity, by inducing a form of non-heritable phenotypic variability, can increase the probability that at least some individuals will survive in rapidly changing, unpredictable environments. Our results suggest how stochasticity may be an important feature rather than a bug in neural development.

2024-08-08

bioRxiv (preprint)

Predicting Future Actions of Reinforcement Learning Agents

Stephen Chung

Scott Niekum

David Scott Krueger

2024-08-07

rl-conference.cc/RLC/2024/Workshop/RLSW (poster)

Simplifying Constraint Inference with Inverse Reinforcement Learning

Adriana Hugessen

Harley Wiltzer

Glen Berseth

2024-08-07

rl-conference.cc/RLC/2024/Workshop/RLSW (poster)

Critical dynamics in spontaneous EEG predict anesthetic-induced loss of consciousness and perturbational complexity

Charlotte Maschke

Jordan O’Byrne

Michele Angelo Colombo

Melanie Boly

Olivia Gosseries

Steven Laureys

Mario Rosanova

Karim Jerbi

Stefanie Blain-Moraes

2024-08-05

Communications Biology (published)

Learning Hybrid Interpretable Models: Theory, Taxonomy, and Methods

Julien Ferry

gabriel laberge

Ulrich Aivodji

A hybrid model involves the cooperation of an interpretable model and a complex black box. At inference, any input of the hybrid model is as… (see more)signed to either its interpretable or complex component based on a gating mechanism. The advantages of such models over classical ones are two-fold: 1) They grant users precise control over the level of transparency of the system and 2) They can potentially perform better than a standalone black box since redirecting some of the inputs to an interpretable model implicitly acts as regularization. Still, despite their high potential, hybrid models remain under-studied in the interpretability/explainability literature. In this paper, we remedy this fact by presenting a thorough investigation of such models from three perspectives: Theory, Taxonomy, and Methods. First, we explore the theory behind the generalization of hybrid models from the Probably-Approximately-Correct (PAC) perspective. A consequence of our PAC guarantee is the existence of a sweet spot for the optimal transparency of the system. When such a sweet spot is attained, a hybrid model can potentially perform better than a standalone black box. Secondly, we provide a general taxonomy for the different ways of training hybrid models: the Post-Black-Box and Pre-Black-Box paradigms. These approaches differ in the order in which the interpretable and complex components are trained. We show where the state-of-the-art hybrid models Hybrid-Rule-Set and Companion-Rule-List fall in this taxonomy. Thirdly, we implement the two paradigms in a single method: HybridCORELS, which extends the CORELS algorithm to hybrid modeling. By leveraging CORELS, HybridCORELS provides a certificate of optimality of its interpretable component and precise control over transparency. We finally show empirically that HybridCORELS is competitive with existing hybrid models, and performs just as well as a standalone black box (or even better) while being partly transparent.

2024-08-05

TMLR (accepted)