Golnoosh Farnadi

Biographie

Golnoosh Farnadi est professeure associée à l'École d'informatique de l'Université McGill et professeure associée à l'Université de Montréal. Elle est membre académique principal à Mila - Institut québécois d'intelligence artificielle et est titulaire d'une chaire CIFAR d'intelligence artificielle au Canada.

Mme Farnadi a fondé le laboratoire EQUAL à Mila / Université McGill, dont elle est l'une des principales chercheuses. Le laboratoire EQUAL (EQuity & EQuality Using AI and Learning algorithms) est un laboratoire de recherche de pointe dédié à l'avancement des domaines de l'équité algorithmique et de l'IA responsable.

Étudiants actuels

Mina Arzaghi

Doctorat - HEC

mina.arzaghi@mila.quebec

florian.carichon@mila.quebec

Florian Carichon

Postdoctorat - McGill

Khaoula Chehbouni

Doctorat - McGill

Co-superviseur⋅e :

Doctorat - McGill

Co-superviseur⋅e :

Pablo Piantanida

clea.chataigner@mila.quebec

Alireza Dehghanpour Farashah

Doctorat - McGill

Co-superviseur⋅e :

alireza.farashah@mila.quebec

marylou.fauchard@mila.quebec

Marylou Fauchard

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

Margarida Carvalho

Prakhar Ganesh

Doctorat - McGill

prakhar.ganesh@mila.quebec

aditi.khandelwal@mila.quebec

Aly Kassem

Collaborateur·rice de recherche - UWindsor

aly.kassem@mila.quebec

Doctorat - McGill

Co-superviseur⋅e :

Siva Reddy

soumya.sharma@mila.quebec

Soumya Sharma

Doctorat - McGill

Co-superviseur⋅e :

Adriana Romero Soriano

william.st-arnaud@mila.quebec

Postdoctorat - McGill

zhuan.shi@mila.quebec

William St-Arnaud

Doctorat - UdeM

Co-superviseur⋅e :

Margarida Carvalho

Emma Tomiuk Tomiuk

Maîtrise recherche - McGill

emma.tomiuk@mila.quebec

Publications

TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models

Alireza Farashah

Text-to-image diffusion models can be misused to generate harmful content through adversarial or paraphrased prompts that bypass built-in sa… (voir plus)fety mechanisms. Existing concept erasure methods often suffer from limited robustness against adversarial prompts, degradation of benign generation quality, or reliance on inference-time interventions that introduce persistent computational overhead. To address these limitations, we formulate concept erasure as a domain alignment problem in the text representation space. We propose a lightweight Text Encoder Alignment framework (TEA) that fine-tunes only the text encoder while keeping the generative backbone fully frozen. Given concept--anchor prompt pairs, our method trains a discriminator to distinguish token-level representations of concept-containing prompts from those of safe anchor prompts, while updating the text encoder to make these representations indistinguishable. TEA introduces zero inference-time overhead and requires only a small number of fine-tuning steps, making it highly efficient to deploy at scale. Despite this efficiency, TEA achieves state-of-the-art erasure robustness against black-box and white-box adversarial attacks on Stable Diffusion v1.4, while preserving generation quality on benign prompts. Furthermore, TEA is model-agnostic and achieves the lowest attack success rate on Stable Diffusion v3.5, extending concept erasure to a Rectified Flow Transformer architecture with T5 conditioning where prior methods remain largely unexplored. Code is available at \href{https://github.com/alirezafarashah/TEA.git}{https://github.com/alirezafarashah/TEA.git}

2026-08-14

arXiv (prépublication)

IP Protection in the Era of Visual Generative AI: A Survey

Alireza Dehghanpour Farashah

Shunchang Liu

Qian Yang

Han Yu

Cao Yang

Chaochao Chen

Yuping Yan

Yaochu Jin

Lingjuan Lyu

The rapid evolution of visual generative AI has introduced a wide range of intellectual property risks, spanning the unauthorized learning, … (voir plus)reproduction, extraction, misuse, and redistribution of protected data and model assets. To address these risks, a growing body of technical defenses has been proposed. However, existing surveys typically organize this literature by lifecycle stage or technical mechanism, which can obscure the protective intent of different methods. This survey presents a two-dimensional taxonomy for IP protection in visual generative models. The primary axis is a Control Logic View, which classifies methods into Information Exposure Control, Generative Behavior Constraint, and Attribution&Accountability according to the risk variable they regulate. The secondary axis distinguishes Data IP from Model IP as cross-cutting asset dimensions. Under this framework, we systematically review protection methods, align evaluation protocols with protection objectives, and discuss open challenges including proactive model-level safeguards, standardized evaluation, robustness against adaptive attacks, and explainable evidence. This survey aims to offer a principled, systematic, and easy-to-follow overview for both new and experienced researchers in visual generative AI IP protection.

2026-08-12

arXiv (prépublication)

Reasoning with Preference Constraints: A Benchmark for Language Models in Many-to-One Matching Markets

2026-08-01

Transactions on Machine Learning Research (accepté)

openreview.net

Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systems

Large Language Models (LLMs)-powered multi-agent systems are increasingly deployed in mixed-motive environments, where agents operate under … (voir plus)asymmetric information and strategic deception due to conflicting or hidden objectives. In these settings, misalignment with collective goals becomes a central concern. We propose a novel framework for evaluating objective misalignment using the social deduction game Werewolf, modifying the objective of a single agent while preserving its assigned role. Across LLMs from four different model families and sizes, four player roles, and three objective formulations, we introduce a dual analysis of the agents' internal reasoning and their public cheap-talk behavior (i.e costless, non-binding communication that does not directly affect the agents' utilities), complemented by an analysis of game outcomes. Our results show that objective misalignment undermines outcomes in inherently adversarial environments, an effect exacerbated by asymmetric information and specialized roles. While compromised agents consistently develop distinct objective-dependent reasoning strategies, these adaptations remain largely invisible in their public behavior. More broadly, our findings suggest that even subtle objective misalignment can profoundly affect collective decision-making, highlighting the need for effective mitigation strategies for LLM-based multi-agent systems.

2026-07-27

arXiv (prépublication)

Can LLMs Cook Jamaican Couscous? A Study of Cultural Novelty in Recipe Generation

Florian Carichon

Romain Rampa

2026-06-24

ACM Conference on Fairness, Accountability, and Transparency (publié)

Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior

Ali Khalil

Aly M. Kassem

Mohamed Abdelrazek

Santu Rana

We investigate whether harmful chain-of-thought (CoT) traces from compromised language models can transfer unsafe behaviour and be distilled… (voir plus) into reusable jailbreak attacks. Using an emergent-misalignment organism and a refusal-ablated jailbroken organism, we transplant harmful CoTs into

2026-06-17

arXiv (prépublication)

IDP-Bench: Benchmarking ability of LLMs to protect personal information in interdependent privacy contexts

Ayana Hussain

Soumya Sharma

Nicholas Vincent

Héber Hwang Arcolezi

Ulrich Aivodji

Large language models (LLMs) are becoming widely deployed as personal AI assistants with access to sensitive user data, making privacy a maj… (voir plus)or challenge for their design and evaluation. Prior work focuses mainly on individual-level risks, overlooking \textbf{interdependent privacy (IDP)}--where one person's data may be revealed by others without their knowledge or consent. We address this gap by introducing \textbf{IDP-Bench}: the first LLM benchmark for IDP scenarios, grounded in the Contextual Integrity (CI) framework. We evaluate eight open-source LLMs on their understanding of IDP scenarios across three levels of IDP reasoning using two LLM judges. Results show strong co-ownership recognition (6/8 models exceed 90\%) but persistent weaknesses in identifying CI parameters (information attribute, primary subject) and IDP-specific parameters such as secondary subjects, where 7/8 models score below 74\%. Models also struggle to judge sharing appropriateness (5/8 scoring below 77\%). While the ability to judge the appropriateness of sharing improves with scale, performance tends to decline in smaller models, and prompt sensitivity remains high on IDP-specific questions--highlighting the need for more targeted study of IDP in LLM privacy research. Data \& code available \href{https://github.com/tisl-lab/Interdependent_Privacy_Bench}{here}.

2026-06-05

arXiv (prépublication)

Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs

Omar Mahmoud

Aly M. Kassem

Thommen George Karimpanal

Buddhika Laknath Semage

Santu Rana

Backdoor attacks in large language models (LLMs) are often treated as isolated trigger-response failures, motivating defenses tailored to sp… (voir plus)ecific triggers or behaviors. We show this view is incomplete. Across diverse backdoor behaviors, we identify a shared latent mechanism that can be detected, causally controlled, and suppressed. Using sparse autoencoders (SAEs) on residual-stream activations, we find a small set of latent features consistently activated across jailbreaking, refusal manipulation, password-locking, bias induction, sentiment misclassification, and country-conditioned harmful advice. These features generalize across Qwen3, Gemma~3, and Llama~3.1 models from 4B to 32B parameters, and across both fine-tuning and weight-editing attacks. Through bidirectional activation steering, we show these features are causal: suppressing them reduces attack success, while amplifying them induces target behaviors on clean prompts. We further train lightweight SAE-feature classifiers that generalize zero-shot to unseen backdoors and outperform residual-stream and weight-diffing baselines. Finally, we introduce Concept Ablation Fine-Tuning (CAFT), which suppresses backdoor formation by ablating the shared latent subspace during training. Together, our results suggest that many backdoors rely on a transferable latent mechanism, enabling unified detection and mitigation.

2026-06-05

arXiv (prépublication)

IDEAFix: Evaluation Framework for Creative Defixation Prompting in LLMs

Florian Carichon

Soumya Sharma

Meaghan Girard

Romain Rampa

Large language models (LLMs) are increasingly used for tasks involving creative problem solving and idea generation. However, there is a lac… (voir plus)k of consensus concerning their creative capabilities: some studies report superior performances compared to humans, while others highlight structural limitations such as fixation and the homogenization of outputs. Existing evaluation approaches either rely on narrow, decontextualized tasks that do not capture goal-oriented generation or on broader settings that confound multiple aspects of the creative process, making it difficult to isolate the effects of task formulation, prompting, and evaluation design. Significantly, the role of structured prompting strategies in shaping idea generation remains underexplored. Therefore, we introduce IDEAFix, an evaluation framework for analyzing divergent thinking in open-ended idea generation tasks. We prompt models to generate multiple original solutions to controlled variations of short design scenarios, task attributes, and defixation prompting strategies. This design enables systematic analysis of how structured guidance influences LLMs' idea generation. Our results show that both task formulation and attribute selection significantly affect models' performance, and that simple prompting strategies can boost the originality of solutions. However, we also observe persistent output homogenization across models, confirming inherent limits in their ability to generate diverse solutions. Overall, IDEAFix provides a controlled, extensible framework for studying the mechanisms underlying LLMs' creativity.

2026-05-29

arXiv (prépublication)

Leveraging Routing Dynamics in Mixture-of-Experts Models for Efficient Language Adaptation

Mixture-of-Experts (MoE) models are widely used to scale language models, yet their expert routing behavior and adaptation in a multilingual… (voir plus) setting remain underexplored. In this work, we study multilingual routing dynamics during continual pre-training of an English-centric MoE model on a multilingual corpus, analyzing how expert usage varies across languages. We find that continual multilingual pre-training leads to diffused, language-agnostic routing in early and middle layers, with language specialization primarily emerging in the final layers. We also show that token-level vocabulary overlap between languages plays an important role in how languages are routed. Motivated by these findings, we propose a parameter-efficient adaptation strategy that updates language-specific and shared experts in the final MoE layers. Experiments on MultiBLiMP and Belebele show that our method achieves a strong performance-efficiency trade-off, attaining competitive performance relative to fine-tuning complete final layers, while updating less than 2% of the parameters. Overall, our findings provide insights into where and how language specialization emerges in MoEs during continual pre-training and provide practical insights for low-resource multilingual adaptation. Our code is available at https://github.com/aditi184/moe-routing-adaptation.

2026-05-27

arXiv (prépublication)

Neighbor-Aware Localized Concept Erasure in Text-to-Image Diffusion Models

Alireza Dehghanpour Farashah

Rik de Vries

Concept erasure in text-to-image diffusion models seeks to remove undesired concepts while preserving overall generative capability. Localiz… (voir plus)ed erasure methods aim to restrict edits to the spatial region occupied by the target concept. However, we observe that suppressing a concept can unintentionally weaken semantically related neighbor concepts, reducing fidelity in fine-grained domains. We propose Neighbor-Aware Localized Concept Erasure (NLCE), a training-free framework designed to better preserve neighboring concepts while removing target concepts. It operates in three stages: (1) a spectrally-weighted embedding modulation that attenuates target concept directions while stabilizing neighbor concept representations, (2) an attention-guided spatial gate that identifies regions exhibiting residual concept activation, and (3) a spatially-gated hard erasure that eliminates remaining traces only where necessary. This neighbor-aware pipeline enables localized concept removal while maintaining the surrounding concept neighborhood structure. Experiments on fine-grained datasets (Oxford Flowers, Stanford Dogs) show that our method effectively removes target concepts while better preserving closely related categories. Additional results on celebrity identity, explicit content and artistic style demonstrate robustness and generalization to broader erasure scenarios.

2026-03-26

arXiv (prépublication)

DELTA-CROSSCODER: ROBUST CROSSCODER IN NARROW FINE-TUNING REGIMES

Aly M. Kassem

Thomas Jiralerspong