Publications

Refining SARS-CoV-2 Intra-host Variation by Leveraging Large-scale Sequencing Data

Fatima Mostefai

Jean-Christophe Grenier

Raphael Poujol

Julie Hussin

2024-05-01

bioRxiv (prépublication)

doi.org

Robust Data-driven Prescriptiveness Optimization

Mehran Poursoltani

Érick Delage

Angelos Georghiou

2024-05-01

ICML.cc/2024/Conference (poster)

doi.org

openreview.net

Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features

Aleksandr Beznosikov

David Dobre

Gauthier Gidel

2024-05-01

ICML.cc/2024/Conference (poster)

doi.org

openreview.net

A self-attention-based CNN-Bi-LSTM model for accurate state-of-charge estimation of lithium-ion batteries

Zeinab Sherkatghanad

Amin Ghazanfari

Vladimir Makarenkov

2024-05-01

Journal of Energy Storage (publié)

doi.org

SelfIE: Self-Interpretation of Large Language Model Embeddings

Haozhe Chen

Carl Vondrick

Chengzhi Mao

How do large language models (LLMs) obtain their answers? The ability to explain and control an LLM's reasoning process is key for reliabili… (voir plus)ty, transparency, and future model developments. We propose SelfIE (Self-Interpretation of Embeddings), a framework that enables LLMs to interpret their own embeddings in natural language by leveraging their ability to respond to inquiries about a given passage. Capable of interpreting open-world concepts in the hidden embeddings, SelfIE reveals LLM internal reasoning in cases such as making ethical decisions, internalizing prompt injection, and recalling harmful knowledge. SelfIE's text descriptions on hidden embeddings also open up new avenues to control LLM reasoning. We propose Supervised Control, which allows editing open-ended concepts while only requiring gradient computation of individual layer. We extend RLHF to hidden embeddings and propose Reinforcement Control that erases harmful knowledge in LLM without supervision targets.

2024-05-01

ICML.cc/2024/Conference (poster)

doi.org

openreview.net

Stealing part of a production language model

Nicholas Carlini

Daniel Paleka

Krishnamurthy Dj Dvijotham

Thomas Steinke

Jonathan Hayase

A. Feder Cooper

Katherine Lee

Matthew Jagielski

Milad Nasr

Arthur Conmy

Eric Wallace

David Rolnick

Florian Tramèr

We introduce the first model-stealing attack that extracts precise, nontrivial information from black-box production language models like … (voir plus)OpenAI's ChatGPT or Google's PaLM-2. Specifically, our attack recovers the embedding projection layer (up to symmetries) of a transformer model, given typical API access. For under \\

2024-05-01

ICML.cc/2024/Conference (présentation orale)

doi.org

openreview.net

Stochastic positional embeddings improve masked image modeling

Amir Bar

Florian Bordes

Assaf Shocher

Mahmoud Assran

Pascal Vincent

Nicolas Ballas

Trevor Darrell

Amir Globerson

Yann LeCun

Masked Image Modeling (MIM) is a promising self-supervised learning approach that enables learning from unlabeled images. Despite its recent… (voir plus) success, learning good representations through MIM remains challenging because it requires predicting the right semantic content in accurate locations. For example, given an incomplete picture of a dog, we can guess that there is a tail, but we cannot determine its exact location. In this work, we propose to incorporate location uncertainty into MIM by using stochastic positional embeddings (StoP). Specifically, we condition the model on stochastic masked token positions drawn from a Gaussian distribution. StoP reduces overfitting to location features and guides the model toward learning features that are more robust to location uncertainties. Quantitatively, StoP improves downstream MIM performance on a variety of downstream tasks, including

2024-05-01

ICML.cc/2024/Conference (poster)

openreview.net