Gauthier Gidel

sadhana.anand@mila.quebec

Biography

I am an assistant professor in the Department of Computer Science and Operations Research (DIRO) at Université de Montréal, a core academic member of Mila – Quebec Artificial Intelligence Institute, and a Canada CIFAR AI Chair.

Previously, I was awarded a Borealis AI Graduate Fellowship, worked at DeepMind and Element AI, and was a Long-Term Visitor at the Simons Institute at UC Berkeley.

My research interests lie at the intersection of game theory, optimization and machine learning.

Current Students

Sadhana Anand

Master's Research - Université de Montréal

yuan-chen.chang@mila.quebec

Yuan-Chen Chang

PhD - Université de Montréal

David Dobre

PhD - Université de Montréal

david-a.dobre@mila.quebec

damien.ferbach@mila.quebec

Nouha Dziri

Independent visiting researcher - N/A

nouha.dziri@gmail.com

PhD - Université de Montréal

Co-supervisor :

Courtney Paquette

niklas.herbster@gmail.com

Niklas Herbster

Research Intern - Université de Montréal

Principal supervisor :

Irina Rish

marco.jiralerspong@mila.quebec

Marco Jiralerspong

PhD - Université de Montréal

andjela.mladenovic@mila.quebec

Zichu Liu

PhD - Université de Montréal

Co-supervisor :

Ioannis Mitliagkas

zichu.liu@mila.quebec

Andjela Mladenovic

PhD - Université de Montréal

Co-supervisor :

Aaron Courville

mehrnaz.mofakhami@mila.quebec

Mehrnaz Mofakhami

Collaborating researcher - Université de Montréal

Co-supervisor :

Ioannis Mitliagkas

Collaborating researcher - Université de Montréal

ailin.rasteh@mila.quebec

Independent visiting researcher - Technical Univeristy of Munich

leo.schwinn@mila.quebec

Tom Stanic

Master's Research - Université de Montréal

tom.stanic@mila.quebec

tommaso.tosato@mila.quebec

Tommaso Tosato

Postdoctorate - Université de Montréal

PhD - Université de Montréal

danilo.vucetic@mila.quebec

PhD - Université de Montréal

Co-supervisor :

Jian Tang

lpxhonneux@gmail.com

What Do Synaptic Weight Distributions Tell Us About Learning in the Brain ?

Blog Posts

June 13, 2024

Roman Pogodin

Jonathan Cornford

Arna Ghosh

Gauthier Gidel

Guillaume Lajoie

Blake Richards

Read the article

Publications

A Generative Approach to LLM Harmfulness Mitigation with Red Flag Tokens

2025-07-24

SoLaR @ Conference on Language Modeling (poster)

Discrete Compositional Generation via General Soft Operators and Robust Reinforcement Learning

Marco Jiralerspong

Esther Derman

Danilo Vucetic

Nikolay Malkin

Bilun Sun

Tianyu Zhang

Pierre-Luc Bacon

A major bottleneck in scientific discovery consists of narrowing an exponentially large set of objects, such as proteins or molecules, to a … (see more)small set of promising candidates with desirable properties. While this process can rely on expert knowledge, recent methods leverage reinforcement learning (RL) guided by a proxy reward function to enable this filtering. By employing various forms of entropy regularization, these methods aim to learn samplers that generate diverse candidates that are highly rated by the proxy function. In this work, we make two main contributions. First, we show that these methods are liable to generate overly diverse, suboptimal candidates in large search spaces. To address this issue, we introduce a novel unified operator that combines several regularized RL operators into a general framework that better targets peakier sampling distributions. Secondly, we offer a novel, robust RL perspective of this filtering process. The regularization can be interpreted as robustness to a compositional form of uncertainty in the proxy function (i.e., the true evaluation of a candidate differs from the proxy's evaluation). Our analysis leads us to a novel, easy-to-use algorithm we name trajectory general mellowmax (TGM): we show it identifies higher quality, diverse candidates than baselines in both synthetic and real-world tasks. Code: https://github.com/marcojira/tgm.

2025-06-19

ArXiv (preprint)

Jailbreak Distillation: Renewable Safety Benchmarking

Jingyu Zhang

Ahmed Elgohary

Xiawei Wang

A S M Iftekhar

Ahmed Magooda

Benjamin Van Durme

Daniel Khashabi

Kyle Jackson

JBDistill Benchmark JBDistill Benchmark

Marah Ihab Abdin

Jyoti Aneja

Harkirat Singh Behl

Sébastien Bubeck

Ronen Eldan

S. Gunasekar

Michael Harrison

Russell J. Hewett

Mojan Javaheripi

Piero Kauffmann

James R. Lee … (see 484 more)

Yin Tat Lee

Yuanzhi Li

Weishung Liu

C. C. T. Mendes

Anh Nguyen

Eric Price

Gustavo de Rosa

Olli Saarikivi

Adil Salim

Tim Beyer

Simon Geisler

Stephan Günnemann. 2025

Blake Bullwinkel

Amanda Minnich

Shiven Chawla

Gary Lopez

Martin Pouliot

Whitney Maxwell

Patrick Chao

Edoardo Debenedetti

Alexander Robey

Maksym Andriushchenko

Francesco Croce

Vikash Sehwag

Edgar Dobriban

Nicolas Flammarion

George J. Pappas

Florian Tramèr

Hamed Hassani

Eric Wong

Jailbreakbench

Zora Che

Stephen Casper

Robert Kirk

Anirudh Satheesh

Stewart Slocum

Lev E McKinney

Rohit Gandikota

Aidan Ewart

Domenic Rosati

Zichu Wu

Zikui Cai

Daya Guo

Dejian Yang

Haowei Zhang

Jun-Mei Song

Ruoyu Zhang

Runxin Xu

Qihao Zhu

Shirong Ma

Peiyi Wang

Xiaoling Bi

Xiaokang Zhang

Xingkai Yu

Yu Wu

Z. F. Wu

Zhibin Gou

Zhihong Shao

Zhuoshu Li

Ziyi Gao

A. Liu

Bing Xue

Bingxuan Wang

Bo WU

Bei Feng

Chenggang Lu

Chenggang Zhao

Chengqi Deng

Chenyu Zhang

C. Ruan

Damai Dai

Deli Chen

Dong-Li Ji

Erhang Li

Fangyun Lin

Fucong Dai

Fuli Luo

Guangbo Hao

Guanting Chen

Guowei Li

Han Bao

Hanwei Xu

Haocheng Wang

Honghui Ding

Huajian Xin

Huazuo Gao

Hui Qu

Hui Li

Jianzhong Guo

Jiashi Li

Jiawei Wang

Jingchang Chen

Jingyang Yuan

Junjie Qiu

Junlong Li

Jinbo Cai

Jia Ni

Jian Liang

Jin Chen

Kai Dong

Kai Hu

Kaige Gao

Kang Guan

Kexin Huang

Kuai Yu

Lean Wang

Lecong Zhang

Liang Zhao

Litong Wang

Liyue Zhang

Lei Xu

Leyi Xia

Mingchuan Zhang

Minghua Zhang

Min Tang

Meng Li

Miaojun Wang

Mingming Li

Ning Tian

Panpan Huang

Meng Wang

Qiancheng Wang

Qinyu Chen

Qiushi Du

Ruiqi Ge

Ruisong Zhang

Ruizhe Pan

Runji Wang

R. J. Chen

Rong Jin

Ruyi Chen

Shanghao Lu

Shangyan Zhou

Shanhuang Chen

Shengfeng Ye

Shiyu Wang

Shuiping Yu

Shunfeng Zhou

Shuting Pan

S. S. Li

Shuang Zhou

Shao-Ping Wu

Tao Yun

Tian Pei

Tianyu Sun

T. Wang

Wangding Zeng

Wanjia Zhao

Wen Liu

Wenfeng Liang

Wenjun Gao

Wen-Xuan Yu

Wentao Zhang

Wei Xiao

Wei An

Xiaodong Liu

Xiaohan Wang

Xiaokang Chen

Xiaotao Nie

Xin Cheng

Jian Li

Xinfeng Xie

Xingchao Liu

Xinyu Yang

Xinyuan Li

Xuecheng Su

Xuheng Lin

Xiangyu Jin

Xi-Cheng Shen

Xiaosha Chen

Xiaowen Sun

Xiaoxi-ang Wang

Xinnan Song

Xinyi Zhou

Xianzu Wang

Xinxia Shan

Y. K. Li

Y. Q. Wang

Y. X. Wei

Yang Zhang

Yan-Hong Xu

Yao Zhao

Yaofeng Sun

Yaohui Wang

Yi Yu

Yichao Zhang

Yifan Shi

Yi Xiong

Ying He

Yishi Piao

Yisong Wang

Yi Chern Tan

Yiyang Ma

Yiyuan Liu

Yongqiang Guo

Yuan Ou

Yuduan Wang

Yue Gong

Yuheng Zou

Yuzi He

Yunfan Xiong

Yuxiang Luo

Yuxiang You

Yu-mei You

Yuxuan Liu

Yuyang Zhou

Y. X. Zhu

Yanping Huang

Yaohui Li

Yang Li

Yi Zheng

Yunxiang Ma

Ying Tang

Yukun Zha

Yuting Yan

Z. Z. Ren

Zehui Ren

Zhangli Sha

Zhe Fu

Zhean Xu

Zhenda Xie

Zhengyan Zhang

Zhewen Hao

Zhicheng Ma

Zhigang Yan

Zhiyu Wu

Zihui Gu

Zijia Zhu

Zijun Liu

Zi-An Li

Ziwei Xie

Ziyang Song

Deep Ganguli

Liane Lovitt

Jackson Kernion

Amanda Askell

Yuntao Bai

Saurav Kadavath

Benjamin Mann

Ethan Perez

Nicholas Schiefer

Kamal Ndousse

Andy Jones

Sam Bowman

Anna Chen

Tom Con-erly

Nova Dassarma

Dawn Drain

Nelson Elhage Sheer

Stanislav Fort

Zac Hatfield-Dodds

T. Henighan

Danny Hernandez

Tristan Hume

Josh Jacobson

Scott Johnston

Shauna Kravec

Catherine Olsson

Sam Ringer

Eli Tran-Johnson

Dario Amodei

Tom Brown

Nicholas Joseph

Sam McCandlish

Chris Olah

Jared Kaplan

Jack Clark. 2022. Red

Aaron Grattafiori

Abhimanyu Dubey

Abhinav Jauhri

Abhinav Pandey

Abhishek Kadian

Ahmad Al-Dahle

Aiesha Letman

Akhil Mathur

Alan Schel-ten

Alex Vaughan

Amy Yang

Angela Fan

Anirudh Goyal

A. Hartshorn

Aobo Yang

Archi Mitra

Archie Sravankumar

Artem Korenev

Arthur Hinsvark

Arun Rao

Aston Zhang

Aurelien Ro-driguez

Austen Gregerson

Ava Spataru

Baptiste Rozière

Bethany Biron

Binh Tang

Bobbie Chern

Charlotte Caucheteux

Chaya Nayak

Chloe Bi

Chris Marra

Chris McConnell

Christian Keller

Christophe Touret

Chunyang Wu

Corinne Wong

Cris-tian Cantón Ferrer

Cyrus Nikolaidis

Damien Al-lonsius

Daniel Song

Danielle Pintz

Danny Livshits

Danny Wyatt

David Esiobu

Dhruv Choudhary

Dhruv Mahajan 0001

Diego Garcia-Olano

Diego Perino

Dieuwke Hupkes

Egor Lakomkin

Ehab A. AlBadawy

Elina Lobanova

Emily Dinan

Eric Michael Smith

Filip Radenovic

Francisco Guzmán

Frank Zhang

Gabriele Synnaeve

Gabrielle Lee

Georgia Lewis

G. Thattai

Graeme Nail

Gregoire Mi-alon

Guan Pang

Guillem Cucurell

Hailey Nguyen

Han-nah Korevaar

Hu Xu

Hugo Touvron

Imanol Iliyan Zarov

Arrieta Ibarra

Is-abel Kloumann

Ishan Misra

Ivan Evtimov

Jack Zhang

Jade Copet

Jaewon Lee

Jan Geffert

Jana Vranes

Jason Park

Jay Mahadeokar

Jeet Shah

Jelmer van der Linde

Jennifer Billock

Jenny Hong

Jenya Lee

Jeremy Fu

J. Fu

Jianfeng Chi

Jianyu Huang

Jiawen Liu

Jie Wang

Jiecao Yu

Joanna Bitton

Joe Spisak

Jongsoo Park

Joseph Rocca

J. Johnstun

Joshua Saxe

Junteng Jia

Kalyan Vasuden Alwala

Karthik Prasad

Kartikeya Upasani

Kate Plawiak

Keqian Li

Kenneth Heafield

Kevin R. Stone

Khalid El-Arini

Krithika Iyer

Kshitiz Malik

Kuen-ley Chiu

Kunal Bhalla

Kushal Lakhotia

Lauren Rantala-Yeary

Laurens van der Maaten

Lawrence Chen

Liang Tan

Liz Jenkins

Louis Martin

Lovish Madaan

Lubo Malo

Lukas Blecher

Lukas Landzaat

Luke de Oliveira

Madeline Muzzi

Mahesh Pasupuleti

Mannat Singh

Manohar Paluri

Marcin Kardas

Maria Tsimpoukelli

Mathew Oldham

Mathieu Rita

Maya Pavlova

Melanie Kam-badur

Mike Lewis

Mitesh Min Si

Kumar Singh

Mona Hassan

Naman Goyal

Narjes Torabi

Niko-lay Bashlykov

Nikolay Bogoychev

Niladri S. Chatterji

Ning Zhang

Olivier Duchenne

Onur Çelebi

Patrick Alrassy

Petar Pengwei Li

Peter Weng

Prajjwal Bhargava

Pratik Dubal

Punit Praveen Krishnan

Singh Koura

Puxin Xu

Qing He

Qingxiao Dong

Ragavan Srinivasan

Raj Ganapathy

Ramon Calderer

Ricardo Silveira Cabral

Robert Stojnic

Roberta Raileanu

Rohan Maheswari

Rohit Girdhar

Rohit Patel

Ro-main Sauvestre

Ron-nie Polidoro

Roshan Sumbaly

Ross Taylor

Ruan Silva

Rui Hou

Rui Wang

S. Hosseini

Sa-hana Chennabasappa

Sanjay Singh

Sean Bell

Seo-hyun Sonia Kim

Sergey Edunov

Shaoliang Nie

Sharan Narang

Sharath Chandra Raparthy

Sheng Shen

Shengye Wan

Shruti Bhosale

Shun Zhang

Simon Van-denhende

Soumya Batra

Spencer Whitman

Sten Sootla

Stephane Collot

Suchin Gururangan

S. Borodinsky

Tamar Herman

Tara Fowler

Tarek Sheasha

Thomas Georgiou

Thomas Scialom

Tobias Speckbacher

Todor Mihaylov

Tong Xiao

Ujjwal Karn

Vedanuj Goswami

Vibhor Gupta

Vignesh Ramanathan

Viktor Kerkez

Vincent Gonguet

Vir-ginie Do

Vish Vogeti

Vitor Albiero

Vladan Petro-vic

Weiwei Chu

Wenhan Xiong

Wenyin Fu

2025-05-27

ArXiv (preprint)

Performative Prediction on Games and Mechanism Design

Fernando P. Santos

2025-04-22

Proceedings of The 28th International Conference on Artificial Intelligence and Statistics (published)

proceedings.mlr.press

Solving Hidden Monotone Variational Inequalities with Surrogate Losses

Junhyung Lyle Kim

Deep learning has proven to be effective in a wide variety of loss minimization problems. However, many applications of interest, like minim… (see more)izing projected Bellman error and min-max optimization, cannot be modelled as minimizing a scalar loss function but instead correspond to solving a variational inequality (VI) problem. This difference in setting has caused many practical challenges as naive gradient-based approaches from supervised learning tend to diverge and cycle in the VI case. In this work, we propose a principled surrogate-based approach compatible with deep learning to solve VIs. We show that our surrogate-based approach has three main benefits: (1) under assumptions that are realistic in practice (when hidden monotone structure is present, interpolation, and sufficient optimization of the surrogates), it guarantees convergence, (2) it provides a unifying perspective of existing methods, and (3) is amenable to existing deep learning optimizers like ADAM. Experimentally, we demonstrate our surrogate-based approach is effective in min-max optimization and minimizing projected Bellman error. Furthermore, in the deep reinforcement learning case, we propose a novel variant of TD(0) which is more compute and sample efficient.

2025-04-22

International Conference on Learning Representations (Accept (Poster))

A Generative Approach to LLM Harmfulness Detection with Red Flag Tokens

Most safety training methods for large-language models (LLMs) based on fine-tuning rely on dramatically changing the output distribution of … (see more)the model when faced with a harmful request, shifting it from an unsafe answer to a refusal to respond. These methods inherently compromise model capabilities and might make auto-regressive models vulnerable to attacks that make likely an initial token of affirmative response. To avoid that, we propose to expand the model's vocabulary with a special token we call a *red flag token* (

2025-03-04

ICLR.cc/2025/Workshop/BuildingTrust (accepted)

LLM-Safety Evaluations Lack Robustness

Tim Beyer

Simon Geisler

Stephan Günnemann

In this paper, we argue that current safety alignment research efforts for large language models are hindered by many intertwined sources of… (see more) noise, such as small datasets, methodological inconsistencies, and unreliable evaluation setups. This can, at times, make it impossible to evaluate and compare attacks and defenses fairly, thereby slowing progress. We systematically analyze the LLM safety evaluation pipeline, covering dataset curation, optimization strategies for automated red-teaming, response generation, and response evaluation using LLM judges. At each stage, we identify key issues and highlight their practical impact. We also propose a set of guidelines for reducing noise and bias in evaluations of future attack and defense papers. Lastly, we offer an opposing perspective, highlighting practical reasons for existing limitations. We believe that addressing the outlined problems in future research will improve the field's ability to generate easily comparable results and make measurable progress.

2025-03-03

ArXiv (preprint)

Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectives

Yan Scholten

Tom Wollschlager

Stephen Casper

Stephan Günnemann

2025-02-16

ArXiv (preprint)

Advantage Alignment Algorithms

The growing presence of artificially intelligent agents in everyday decision-making, from LLM assistants to autonomous vehicles, hints at a … (see more)future in which conflicts may arise from each agent optimizing individual interests. In general-sum games these conflicts are apparent, where naive Reinforcement Learning agents get stuck in Pareto-suboptimal Nash equilibria. Consequently, opponent shaping has been introduced as a method with success at finding socially beneficial equilibria in social dilemmas. In this work, we introduce Advantage Alignment, a family of algorithms derived from first principles that perform opponent shaping efficiently and intuitively. This is achieved by aligning the advantages of conflicting agents in a given game by increasing the probability of mutually-benefiting actions. We prove that existing opponent shaping methods, including LOLA and LOQA, implicitly perform Advantage Alignment. Compared to these works, Advantage Alignment mathematically simplifies the formulation of opponent shaping and seamlessly works for continuous action domains. We also demonstrate the effectiveness of our algorithm in a wide range of social dilemmas, achieving state of the art results in each case, including a social dilemma version of the Negotiation Game.

2025-01-21

International Conference on Learning Representations (oral)

Learning Diverse Attacks on Large Language Models for Robust Red-Teaming and Safety Tuning

Juho Lee

Sung Ju Hwang

Nikolay Malkin

Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe and responsible deployment of lar… (see more)ge language models (LLMs). Developing effective protection against many modes of attack prompts requires discovering diverse attacks. Automated red-teaming typically uses reinforcement learning to fine-tune an attacker language model to generate prompts that elicit undesirable responses from a target LLM, as measured, for example, by an auxiliary toxicity classifier. We show that even with explicit regularization to favor novelty and diversity, existing approaches suffer from mode collapse or fail to generate effective attacks. As a flexible and probabilistically principled alternative, we propose to use GFlowNet fine-tuning, followed by a secondary smoothing phase, to train the attacker model to generate diverse and effective attack prompts. We find that the attacks generated by our method are effective against a wide range of target LLMs, both with and without safety tuning, and transfer well between target LLMs. Finally, we demonstrate that models safety-tuned using a dataset of red-teaming prompts generated by our method are robust to attacks from other RL-based red-teaming approaches.

2025-01-21

International Conference on Learning Representations (poster)

Efficient Adversarial Training in LLMs with Continuous Attacks

Stephan Günnemann

Large language models (LLMs) are vulnerable to adversarial attacks that can bypass their safety guardrails. In many domains, adversarial tra… (see more)ining has proven to be one of the most promising methods to reliably improve robustness against such attacks. Yet, in the context of LLMs, current methods for adversarial training are hindered by the high computational costs required to perform discrete adversarial attacks at each training iteration. We address this problem by instead calculating adversarial attacks in the continuous embedding space of the LLM, which is orders of magnitudes more efficient. We propose a fast adversarial training algorithm (C-AdvUL) composed of two losses: the first makes the model robust on continuous embedding attacks computed on an adversarial behaviour dataset; the second ensures the usefulness of the final model by fine-tuning on utility data. Moreover, we introduce C-AdvIPO, an adversarial variant of IPO that does not require utility data for adversarially robust alignment. Our empirical evaluation on five models from different families (Gemma, Phi3, Mistral, Zephyr, Llama2) and at different scales (2B, 3.8B, 7B) shows that both algorithms substantially enhance LLM robustness against discrete attacks (GCG, AutoDAN, PAIR), while maintaining utility. Our results demonstrate that robustness to continuous perturbations can extrapolate to discrete threat models. Thereby, we present a path toward scalable adversarial training algorithms for robustly aligning LLMs.

2024-12-10

Conference on Neural Information Processing Systems (Accept (spotlight))

Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences

Damien Ferbach

Quentin Bertrand

Avishek Joey Bose

The rapid progress in generative models has resulted in impressive leaps in generation quality, blurring the lines between synthetic and rea… (see more)l data. Web-scale datasets are now prone to the inevitable contamination by synthetic data, directly impacting the training of future generated models. Already, some theoretical results on self-consuming generative models (a.k.a., iterative retraining) have emerged in the literature, showcasing that either model collapse or stability could be possible depending on the fraction of generated data used at each retraining step. However, in practice, synthetic data is often subject to human feedback and curated by users before being used and uploaded online. For instance, many interfaces of popular text-to-image generative models, such as Stable Diffusion or Midjourney, produce several variations of an image for a given query which can eventually be curated by the users. In this paper, we theoretically study the impact of data curation on iterated retraining of generative models and show that it can be seen as an \emph{implicit preference optimization mechanism}. However, unlike standard preference optimization, the generative model does not have access to the reward function or negative samples needed for pairwise comparisons. Moreover, our study doesn't require access to the density function, only to samples. We prove that, if the data is curated according to a reward model, then the expected reward of the iterative retraining procedure is maximized. We further provide theoretical results on the stability of the retraining loop when using a positive fraction of real data at each step. Finally, we conduct illustrative experiments on both synthetic datasets and on CIFAR10 showing that such a procedure amplifies biases of the reward model.

2024-12-10

NEURIPS (Accept (spotlight))