Reihaneh Rabbany

hussein.abdallah@mail.concordia.ca

Biographie

Reihaneh Rabbany est professeure adjointe à l'École d'informatique de l'Université McGill. Elle est membre du corps professoral de Mila – Institut québécois d’intelligence artificielle et titulaire d'une chaire en IA Canada-CIFAR. Elle est également membre du corps enseignant du Centre pour l’étude de la citoyenneté démocratique de McGill. Avant de se joindre à l’Université McGill, elle a été boursière postdoctorale à la School of Computer Science de l'Université Carnegie Mellon. Elle a obtenu un doctorat à l’Université de l’Alberta, au Département d'informatique. Elle dirige le laboratoire de données complexes, dont les recherches se situent à l'intersection de la science des réseaux, de l'exploration des données et de l'apprentissage automatique, et se concentrent sur l'analyse des données interconnectées du monde réel et sur les applications sociales.

Étudiants actuels

Hussein Abdallah

Postdoctorat - McGill

aurelien.buck-kaeffer@mila.quebec

Aurélien Bück-Kaeffer

Maîtrise recherche - McGill

Jacob Chmura

Maîtrise recherche - McGill

Superviseur⋅e principal⋅e :

Nicolas Le Roux

jacob.chmura@mila.quebec

Roxanne Corriveau

Stagiaire de recherche - UdeM

Superviseur⋅e principal⋅e :

mcorriveau303@gmail.com

Johnny Dellas

Stagiaire de recherche - McGill

john.dellas@mila.quebec

Islam Eldifrawi

Visiteur de recherche indépendant - University of Sherbrooke

sulum821991@gmail.com

sophia.garrel@mila.quebec

Aarash Feizi

Doctorat - McGill

Co-superviseur⋅e :

Adriana Romero Soriano

Collaborateur·rice de recherche - McGill

mattia.greiche@mila.quebec

Mattia Greiche

Stagiaire de recherche - McGill

Nazia Hossain

Stagiaire de recherche - McGill

nazia.hossain@mila.quebec

Shenyang Huang

Collaborateur·rice de recherche - McGill

Co-superviseur⋅e :

Anne Imouza

Doctorat - McGill

Superviseur⋅e principal⋅e :

anne.imouza@mila.quebec

Hugh Kelly

Collaborateur·rice de recherche - McGill University

hugh.kelly@mila.quebec

Emma Kondrup

Doctorat - McGill

Co-superviseur⋅e :

Catherine Régis

emma.kondrup@mila.quebec

Andrew Lin

Stagiaire de recherche - McGill University

andrew.lin@mila.quebec

victor.livernoche@mila.quebec

Victor Livernoche

Doctorat - McGill

shahrad.mohammadzadeh@mila.quebec

Sitao Luan

Postdoctorat - McGill

Superviseur⋅e principal⋅e :

Shahrad Mohammadzadeh

Maîtrise recherche - McGill

Co-superviseur⋅e :

Doina Precup

Visiteur de recherche indépendant - McGill

pratheeksha.nair@mila.quebec

Collaborateur·rice alumni - McGill

sahar.omidishayegan@mila.quebec

farimah.poursafaei@mila.quebec

Farimah Poursafaei

Collaborateur·rice alumni - McGill

Maîtrise recherche - McGill University

sebastian.sabry@mila.quebec

Dorsaf Sallami

Collaborateur·rice de recherche - McGill

dorsaf.sallami@mila.quebec

jacob.mila.handle@tianshome.com

Sneheel Sarangi

Maîtrise recherche - McGill

sneheelsarangi@gmail.com

Collaborateur·rice de recherche - McGill University

shiyan.wliu@gmail.com

Jacob-Junqi Tian

Maîtrise recherche - McGill

Collaborateur·rice de recherche - UdeM

Superviseur⋅e principal⋅e :

Guillaume Rabusseau

aaron.tjandra@mila.quebec

Li Wei Wang

Stagiaire de recherche - McGill

liweiwang2002@gmail.com

Tong Wu

Collaborateur·rice de recherche - McGill University

Co-superviseur⋅e :

chin-chen.yang@mila.quebec

Doctorat - McGill

Shirley Zhao

Maîtrise recherche - McGill

xueli.zhao@mila.quebec

Sveta Zhuk

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

svetlana.zhuk@mila.quebec

Billets de blogue

Un groupe hétéroclite de huit jeunes adultes se tient serré sur un toit, souriant et riant avec la silhouette de la ville en arrière-plan. Deux encarts circulaires mettent en évidence une caméra vintage tenue par l'un des membres du groupe, soulignant l'élément deepfake.

16 décembre 2025

Démasquer les deepfakes grâce à l'IA

par

Victor Livernoche

Reihaneh Rabbany

Lire l'article

Flight-SEIR: Incorporating Flight Data to Improve Epidemiological Modelling and Disease Outbreak Prevention

3 août 2021

Flight-SEIR : incorporer les données de vol pour améliorer la modélisation épidémiologique et la prévention d’éclosions de maladies infectieuses

par

Shenyang Huang

Reihaneh Rabbany

Lire l'article

Publications

Hallucination Detox: Sensitivity Dropout (SenD) for Large Language Model Training

Shahrad Mohammadzadeh

Juan David Guerra

Marco Bonizzato

Golnoosh Farnadi

As large language models (LLMs) become increasingly prevalent, concerns about their reliability, particularly due to hallucinations - factua… (voir plus)lly inaccurate or irrelevant outputs - have grown. Our research investigates the relationship between the uncertainty in training dynamics and the emergence of hallucinations. Using models from the Pythia suite and several hallucination detection metrics, we analyze hallucination trends and identify significant variance during training. To address this, we propose Sensitivity Dropout (SenD), a novel training protocol designed to reduce hallucination variance during training by deterministically dropping embedding indices with significant variability. In addition, we develop an unsupervised hallucination detection metric, Efficient EigenScore (EES), which approximates the traditional EigenScore in 2x speed. This metric is integrated into our training protocol, allowing SenD to be both computationally scalable and effective at reducing hallucination variance. SenD improves test-time reliability of Pythia and Meta's Llama models by up to 17% and enhances factual accuracy in Wikipedia, Medical, Legal, and Coding domains without affecting downstream task performance.

2024-12-31

Association for Computational Linguistics (publié)

Online Influence Campaigns: Strategies and Vulnerabilities

Andreea Musulan

Veronica Xia

Ethan Kosak-Hine

Tom Gibbs

Vidya Sujaya

U. Montr'eal

Ivado

M. University

In order to combat the creation and spread of harmful content online, this paper defines and contextualizes the concept of inauthentic, soci… (voir plus)etal-scale manipulation by malicious actors. We review the literature on societally harmful content and how it proliferates to analyze the manipulation strategies used by such actors and the vulnerabilities they target. We also provide an overview of three case studies of extensive manipulation campaigns to emphasize the severity of the problem. We then address the role that Artificial Intelligence plays in the development and dissemination of harmful content, and how its evolution presents new threats to societal cohesion for countries across the globe. Our survey aims to increase our understanding of not just particular aspects of these threats, but also the strategies underlying their deployment, so we can effectively prepare for the evolving cybersecurity landscape.

2024-12-31

arXiv (prépublication)

OpenFake: An Open Dataset and Platform Toward Real-World Deepfake Detection

Akshatha Arodi

Ga'etan Marceau Caron

Deepfakes, synthetic media created using advanced AI techniques, pose a growing threat to information integrity, particularly in politically… (voir plus) sensitive contexts. This challenge is amplified by the increasing realism of modern generative models, which our human perception study confirms are often indistinguishable from real images. Yet, existing deepfake detection benchmarks rely on outdated generators or narrowly scoped datasets (e.g., single-face imagery), limiting their utility for real-world detection. To address these gaps, we present OpenFake, a large politically grounded dataset specifically crafted for benchmarking against modern generative models with high realism, and designed to remain extensible through an innovative crowdsourced adversarial platform that continually integrates new hard examples. OpenFake comprises nearly four million total images: three million real images paired with descriptive captions and almost one million synthetic counterparts from state-of-the-art proprietary and open-source models. Detectors trained on OpenFake achieve near-perfect in-distribution performance, strong generalization to unseen generators, and high accuracy on a curated in-the-wild social media test set, significantly outperforming models trained on existing datasets. Overall, we demonstrate that with high-quality and continually updated benchmarks, automatic deepfake detection is both feasible and effective in real-world settings.

2024-12-31

arXiv.org (prépublication)

PairBench: Are Vision-Language Models Reliable at Comparing What They See?

Aarash Feizi

Sai Rajeswar

Adriana Romero

Spandana Gella

Valentina Zantedeschi

Joao Monteiro

Understanding how effectively large vision language models (VLMs) compare visual inputs is crucial across numerous applications, yet this fu… (voir plus)ndamental capability remains insufficiently assessed. While VLMs are increasingly deployed for tasks requiring comparative judgment, including automated evaluation, re-ranking, and retrieval-augmented generation, no systematic framework exists to measure their performance in these scenarios. We present PairBench, a simple framework that evaluates VLMs as customizable similarity tools using widely available image datasets. Our approach introduces four key metrics for reliable comparison: alignment with human annotations, consistency across pair ordering, distribution smoothness, and controllability through prompting. Our analysis reveals that no model consistently excels across all metrics, with each demonstrating distinct strengths and weaknesses. Most concerning is the widespread inability of VLMs to maintain symmetric similarity scores. Interestingly, we demonstrate that performance on our benchmark strongly correlates with popular benchmarks used for more complex tasks, while providing additional metrics into controllability, smoothness and ordering. This makes PairBench a unique and comprehensive framework to evaluate the performance of VLMs for automatic evaluation depending on the task.

2024-12-31

arXiv.org (prépublication)

The Singapore Consensus on Global AI Safety Research Priorities

Yoshua Bengio

Tegan Maharaj

Luke Ong

Stuart Russell

Dawn Song

Max Tegmark

Lan Xue

Ya-Qin Zhang

Stephen Casper

Wan Sie Lee

Sören Mindermann

Vanessa Wilfred

Vidhisha Balachandran

Fazl Barez

Michael Belinsky

Imane Bello

Malo Bourgon

Mark Brakel

Simeon Campos

Duncan Cass-Beggs … (voir 68 de plus)

Jiahao Chen

Rumman Chowdhury

Kuan Chua Seah

Jeff Clune

Juntao Dai

Agnes Delaborde

Nouha Dziri

Francisco Eiras

Joshua Engels

Jinyu Fan

Adam Gleave

Noah Goodman

Fynn Heide

Johannes Heidecke

Dan Hendrycks

Cyrus Hodes

Bryan Low Kian Hsiang

Minlie Huang

Sami Jawhar

Wang Jingyu

Adam Tauman Kalai

Meindert Kamphuis

Mohan Kankanhalli

Subhash Kantamneni

Mathias Bonde Kirk

Thomas Kwa

Jeffrey Ladish

Kwok-Yan Lam

Wan Lee Sie

Taewhi Lee

Xiaojian Li

Jiajun Liu

Chaochao Lu

Yifan Mai

Richard Mallah

Julian Michael

Nick Moës

Simon Möller

Kihyuk Nam

Kwan Yee Ng

Mark Nitzberg

Besmira Nushi

Seán Ó hÉigeartaigh

Alejandro Ortega

Pierre Peigné

James Petrie

Benjamin Prud'homme

Nayat Sanchez-Pi

Sarah Schwettmann

Buck Shlegeris

Saad Siddiqui

Aradhana Sinha

Martín Soto

Cheston Tan

Dong Ting

William Tjhi

Robert Trager

Brian Tse

Anthony Tung K. H.

Vanessa Wilfred

John Willes

Denise Wong

Wei Xu

Rongwu Xu

Yi Zeng

HongJiang Zhang

Djordje Žikelić

Rapidly improving AI capabilities and autonomy hold significant promise of transformation, but are also driving vigorous debate on how to en… (voir plus)sure that AI is safe, i.e., trustworthy, reliable, and secure. Building a trusted ecosystem is therefore essential – it helps people embrace AI with confidence and gives maximal space for innovation while avoiding backlash. This requires policymakers, industry, researchers and the broader public to collectively work toward securing positive outcomes from AI’s development. AI safety research is a key dimension. Given that the state of science today for building trustworthy AI does not fully cover all risks, accelerated investment in research is required to keep pace with commercially driven growth in system capabilities. Goals: The 2025 Singapore Conference on AI (SCAI): International Scientific Exchange on AI Safety aims to support research in this important space by bringing together AI scientists across geographies to identify and synthesise research priorities in AI safety. The result, The Singapore Consensus on Global AI Safety Research Priorities, builds on the International AI Safety Report-A (IAISR) chaired by Yoshua Bengio and backed by 33 governments. By adopting a defence-in-depth model, this document organises AI safety research domains into three types: challenges with creating trustworthy AI systems (Development), challenges with evaluating their risks (Assessment), and challenges with monitoring and intervening after deployment (Control). Through the Singapore Consensus, we hope to globally facilitate meaningful conversations between AI scientists and AI policymakers for maximally beneficial outcomes. Our goal is to enable more impactful R&D efforts to rapidly develop safety and evaluation mechanisms and foster a trusted ecosystem where AI is harnessed for the public good.

2024-12-31

arXiv (prépublication)

Unified Game Moderation: Soft-Prompting and LLM-Assisted Label Transfer for Resource-Efficient Toxicity Detection

Zachary Yang

Domenico Tullo

Toxicity detection in gaming communities faces significant scaling challenges when expanding across multiple games and languages, particular… (voir plus)ly in real-time environments where computational efficiency is crucial. We present two key findings to address these challenges while building upon our previous work on ToxBuster, a BERT-based real-time toxicity detection system. First, we introduce a soft-prompting approach that enables a single model to effectively handle multiple games by incorporating game-context tokens, matching the performance of more complex methods like curriculum learning while offering superior scalability. Second, we develop an LLM-assisted label transfer framework using GPT-4o-mini to extend support to seven additional languages. Evaluations on real game chat data across French, German, Portuguese, and Russian achieve macro F1-scores ranging from 32.96% to 58.88%, with particularly strong performance in German, surpassing the English benchmark of 45.39%. In production, this unified approach significantly reduces computational resources and maintenance overhead compared to maintaining separate models for each game and language combination. At Ubisoft, this model successfully identifies an average of 50 players, per game, per day engaging in sanctionable behavior.

2024-12-31

arXiv (prépublication)

Higher Order Transformers: Enhancing Stock Movement Prediction On Multimodal Time-Series Data

Soroush Omranpour

Guillaume Rabusseau

In this paper, we tackle the challenge of predicting stock movements in financial markets by introducing Higher Order Transformers, a novel … (voir plus)architecture designed for processing multivariate time-series data. We extend the self-attention mechanism and the transformer architecture to a higher order, effectively capturing complex market dynamics across time and variables. To manage computational complexity, we propose a low-rank approximation of the potentially large attention tensor using tensor decomposition and employ kernel attention, reducing complexity to linear with respect to the data size. Additionally, we present an encoder-decoder model that integrates technical and fundamental analysis, utilizing multimodal signals from historical prices and related tweets. Our experiments on the Stocknet dataset demonstrate the effectiveness of our method, highlighting its potential for enhancing stock movement prediction in financial markets.

2024-12-12

ArXiv (prépublication)

A Simulation System Towards Solving Societal-Scale Manipulation

Maximilian Puelma Touzel

Sneheel Sarangi

Austin Welch

Gayatri K

Dan Zhao

Zachary Yang

Hao Yu

Ethan Kosak-Hine

Tom Gibbs

Andreea Musulan

Camille Thibault

Busra Tugce Gurbuz

The rise of AI-driven manipulation poses significant risks to societal trust and democratic processes. Yet, studying these effects in real-w… (voir plus)orld settings at scale is ethically and logistically impractical, highlighting a need for simulation tools that can model these dynamics in controlled settings to enable experimentation with possible defenses. We present a simulation environment designed to address this. We elaborate upon the Concordia framework that simulates offline, `real life' activity by adding online interactions to the simulation through social media with the integration of a Mastodon server. We improve simulation efficiency and information flow, and add a set of measurement tools, particularly longitudinal surveys. We demonstrate the simulator with a tailored example in which we track agents' political positions and show how partisan manipulation of agents can affect election results.

2024-10-16

ArXiv (prépublication)

Epistemic Integrity in Large Language Models

Bijean Ghafouri

Shahrad Mohammadzadeh

James Zhou

Pratheeksha Nair

Jacob-Junqi Tian

Mayank Goel

Large language models are increasingly relied upon as sources of information, but their propensity for generating false or misleading statem… (voir plus)ents with high confidence poses risks for users and society. In this paper, we confront the critical problem of epistemic miscalibration—where a model's linguistic assertiveness fails to reflect its true internal certainty. We introduce a new human-labeled dataset and a novel method for measuring the linguistic assertiveness of Large Language Models which cuts error rates by over 50% relative to previous benchmarks. Validated across multiple datasets, our method reveals a stark misalignment between how confidently models linguistically present information and their actual accuracy. Further human evaluations confirm the severity of this miscalibration. This evidence underscores the urgent risk of the overstated certainty Large Language Models hold which may mislead users on a massive scale. Our framework provides a crucial step forward in diagnosing and correcting this miscalibration, offering a path to safer and more trustworthy AI across domains.

2024-10-11

NeurIPS.cc/2024/Workshop/SafeGenAi (poster)

Simulation System Towards Solving Societal-Scale Manipulation

Maximilian Puelma Touzel

Sneheel Sarangi

Austin Welch

Gayatri K

Dan Zhao

Zachary Yang

Hao Yu

Tom Gibbs

Ethan Kosak-Hine

Andreea Musulan

Camille Thibault

Busra Tugce Gurbuz

The rise of AI-driven manipulation poses significant risks to societal trust and democratic processes. Yet, studying these effects in real-w… (voir plus)orld settings at scale is ethically and logistically impractical, highlighting a need for simulation tools that can model these dynamics in controlled settings to enable experimentation with possible defenses. We present a simulation environment designed to address this. We elaborate upon the Concordia framework that simulates offline, `real life' activity by adding online interactions to the simulation through social media with the integration of a Mastodon server. Through a variety of means we then improve simulation efficiency and information flow, and add a set of measurement tools, particularly longitudinal surveys of the agents' political positions. We demonstrate the simulator with a tailored example of how partisan manipulation of agents can affect election results.

2024-10-11

NeurIPS.cc/2024/Workshop/SafeGenAi (poster)

The Structural Safety Generalization Problem

Tom Gibbs

Julius Broomfield

George Ingebretsen

Ethan Kosak-Hine

Tia Nasir

Jason Zhang

Reihaneh Iranmanesh

Sara Pieri

It is widely known that AI is vulnerable to adversarial examples, from pixel perturbations to jailbreaks. We propose that there is a key, ea… (voir plus)sier class of problems that is also still unsolved: failures of safety to generalize over structure, despite semantic equivalence. We demonstrate this vulnerability by showing how recent AI systems are differently vulnerable both to multi-turn and multi-image attacks, compared to their single-turn and single-image counterparts with equivalent meaning. We suggest this is the same class of vulnerability as that found in yet unconnected threads of the literature: vulnerabilities to low-resource languages and indefensibility of strongly superhuman Go AIs to cyclic attacks. When viewed together, these reveal a common picture: models that are not only vulnerable to attacks, but vulnerable to attacks with near identical meaning in their benign and harmful components both, and only different in structure. In contrast to attacks with identical benign input (e.g., pictures that look like cats) but unknown semanticity of the harmful component (e.g., diverse noise that is all unintelligible to humans), these represent a class of attacks where semantic understanding and defense against one version should guarantee defense against others—yet current AI safety measures do not. This vulnerability represents a necessary but not sufficient condition towards defending against attacks whose harmful component has arbitrary semanticity. Consequently, by building on the data and approaches we highlight, we frame an intermediate problem for AI safety to solve, that represents a critical checkpoint towards safe AI while being far more tractable than trying to solve it directly and universally.

2024-10-11

NeurIPS.cc/2024/Workshop/SafeGenAi (poster)

Decompose, Recompose, and Conquer: Multi-modal LLMs are Vulnerable to Compositional Adversarial Attacks in Multi-Image Queries

Julius Broomfield

George Ingebretsen

Reihaneh Iranmanesh

Sara Pieri

Ethan Kosak-Hine

Tom Gibbs

Large Language Models have been extensively studied for their vulnerabilities, particularly in the context of adversarial attacks. However, … (voir plus)the emergence of Vision Language Models introduces new modalities of risk that have not yet been thoroughly explored, especially when processing multiple images simultaneously. In this paper, we introduce two black-box jailbreak methods that leverage multi-image inputs to uncover vulnerabilities in these models. We present a new safety evaluation dataset for multimodal LLMs called MultiBench, which is composed of these jailbreak methods. These methods can easily be applied and evaluated using our toolkit. We test these methods against six safety aligned frontier models from Google, OpenAI, and Anthropic, revealing significant safety vulnerabilities. Our findings suggest that even the most powerful language models remain vulnerable against compositional adversarial attacks, specifically those composed of multiple images.

2024-10-08

NeurIPS.cc/2024/Workshop/Red_Teaming_GenAI (poster)