Golnoosh Farnadi

Biographie

Golnoosh Farnadi est professeure associée à l'École d'informatique de l'Université McGill et professeure associée à l'Université de Montréal. Elle est membre académique principal à Mila - Institut québécois d'intelligence artificielle et est titulaire d'une chaire CIFAR d'intelligence artificielle au Canada.

Mme Farnadi a fondé le laboratoire EQUAL à Mila / Université McGill, dont elle est l'une des principales chercheuses. Le laboratoire EQUAL (EQuity & EQuality Using AI and Learning algorithms) est un laboratoire de recherche de pointe dédié à l'avancement des domaines de l'équité algorithmique et de l'IA responsable.

Étudiants actuels

Mina Arzaghi

Doctorat - HEC

mina.arzaghi@mila.quebec

florian.carichon@mila.quebec

Postdoctorat - McGill

Khaoula Chehbouni

Doctorat - McGill

Co-superviseur⋅e :

Doctorat - McGill

Co-superviseur⋅e :

Pablo Piantanida

clea.chataigner@mila.quebec

Alireza Dehghanpour Farashah

Doctorat - McGill

Co-superviseur⋅e :

alireza.farashah@mila.quebec

marylou.fauchard@mila.quebec

Marylou Fauchard

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

Margarida Carvalho

Prakhar Ganesh

Doctorat - McGill

prakhar.ganesh@mila.quebec

aditi.khandelwal@mila.quebec

Aly Kassem

Collaborateur·rice de recherche - UWindsor

aly.kassem@mila.quebec

Doctorat - McGill

Co-superviseur⋅e :

Siva Reddy

mahika.phutane@mila.quebec

Mahika Phutane

Postdoctorat - McGill

ameline.ramesan@mila.quebec

Ameline Ramesan

Maîtrise recherche - McGill

soumya.sharma@mila.quebec

Soumya Sharma

Doctorat - McGill

Co-superviseur⋅e :

Adriana Romero Soriano

william.st-arnaud@mila.quebec

Postdoctorat - McGill

zhuan.shi@mila.quebec

William St-Arnaud

Doctorat - UdeM

Co-superviseur⋅e :

Margarida Carvalho

Emma Tomiuk Tomiuk

Maîtrise recherche - McGill

emma.tomiuk@mila.quebec

Publications

Privacy in Context Is Being Overlooked in the Age of General-Purpose LLMs

Prakhar Ganesh

Yash More

Marco Romanelli

Ferdinando Fioretto

As large language models (LLMs) continue to grow in scope and capability, privacy concerns are on the rise. Since users increasingly interac… (voir plus)t with a single general-purpose system across many distinct contexts, there is a natural tendency to pursue a one-size-fits-all solution to privacy. Unfortunately, this overlooks a crucial insight from decades of privacy research: privacy is inherently contextual. In this paper, we argue that existing work in prompt privacy is drifting towards attempts to design a unified framework, which is highly misleading and can foster a false sense of security. Furthermore, we argue that even future work in this direction needs to wrestle with fundamental tensions across various desirable properties of prompt privacy. Thus, such continued attempts towards a unified framework need to be reconsidered. We recommend three critical paths forward: application-specific deployment of LLMs, context-specific evaluation of privacy frameworks, and better privacy literacy.

2026-10-20

arXiv (prépublication)

Loose Lips: Understanding Privacy Leakage in Agentic Dialogue

Héber H. Arcolezi

Adriana Romero

As LLM agents are increasingly deployed in high-stakes enterprise settings, understanding how they handle sensitive information in agentic d… (voir plus)ialogue has become critical. Privacy leakage in such settings requires reasoning about whether information flows respect the contextual norms under which they originated. We extend CIMemories to CIMEMORIES DIALOGUE, a benchmark of 4,410 two-agent dialogue instances grounded in Contextual Integrity theory. Using this benchmark, we systematically evaluate 7 open-weight models from the Qwen3 and Llama 3.1 families across conversation lengths of 1 and 6 turns under three prompting conditions. We find that task capability and privacy leakage are fundamentally coupled: more capable models are systematically leakier, with no model in our evaluation able to decouple the two. Thinking models consistently leak more than their non-thinking counterparts, a pattern we attribute to the extended reasoning surface created by chain-of-thought. Leakage compounds with conversation length, rising up to 3x from 1 to 6 turns. Our results suggest that the coupling between task capability and privacy leakage is not an artifact of misconfiguration but a structural feature of current agentic systems.

2026-10-06

arXiv (prépublication)

Beyond Fair Answers: Learning Fair and Unlearning Biased Reasoning in LLMs

Mina Arzaghi

Large Language Models (LLMs) have demonstrated strong reasoning capabilities on tasks with well-defined solutions, such as mathematical and … (voir plus)logical reasoning. However, reasoning capability does not necessarily imply unbiased reasoning. In socially ambiguous contexts, where the available evidence is insufficient to support one conclusion over another, models may rely on demographic stereotypes or unsupported social assumptions. Recent work shows that chain-of-thought (CoT) reasoning can contain social stereotypes and that biased reasoning steps are associated with stereotype expression and incorrect predictions. This motivates evaluating fairness in the reasoning process, not only in the final answer. We therefore ask: Can we train LLMs not only to produce fair answers, but to reason fairly? We study this question using a fairness-oriented adaptation of BBQ, where ambiguous cases test whether demographic alternatives receive comparable probability when evidence is insufficient, while disambiguated cases test evidence-grounded accuracy. Following reasoning-trace fine-tuning approaches such as STaR and ReGiFT, we fine-tune instruction-tuned LLMs on reasoning traces generated by stronger models and compare base and fine-tuned models under direct and CoT inference. We evaluate both final decisions and generated reasoning traces, and test generalization on SocioEconomicQA and CrowS-Pairs. Building on this analysis, we propose a reasoning-level fairness framework that jointly learns fair reasoning and unlearns biased reasoning. Evidence-grounded traces form the positive learning set, while traces containing unsupported demographic or stereotype-based inferences form the forget set. Our objective combines supervised fine-tuning on fair traces, a negative objective on biased traces, and KL regularization on unrelated reasoning data to preserve general reasoning ability. We evaluate whether this process improves selection of the fair answer and, more importantly, reduces the probability gap between gendered options when the context provides no evidence favoring either one. We separately measure accuracy in disambiguated contexts to ensure that the model still uses available evidence correctly. At the reasoning level, we evaluate stereotype-dependent inferences and the relative likelihood of fair versus biased reasoning traces. We compare against ReGiFT, standard CoT, and instruction-based reasoning baselines, and evaluate generalization across additional bias benchmarks while monitoring general reasoning performance for capability loss. This work moves fairness intervention from the final decision to the reasoning process that produces it. By combining fair-reasoning supervision, biased-reasoning unlearning, and capability preservation, we investigate whether LLMs can learn not only to give fair answers, but to reason without relying on unsupported demographic stereotypes.

2026-09-29

WiML @ Neural Information Processing Systems (publié)

openreview.net

LLM Persona Unlearning

Kemou Li

Qizhou Wang

Fengpeng Li

Jiantao Zhou

Pre-training equips large language models (LLMs) with a broad repertoire of behavioral patterns associated with roles, styles, values, and g… (voir plus)oals. Post-training teaches conditional enactment and makes a helpful Assistant the default, but it does not erase alternative modes from the weights; explicit prompts can therefore elicit personas that repeatedly shape judgment, language, and action. In open-weight settings, runtime controls can be removed, motivating persona unlearning: a weight-level edit that makes a designated persona difficult to elicit and enact on unseen contexts. We introduce PersonaUnlearnBench, a model-specific paired benchmark spanning six LLMs from three families and five personas, with aligned forget/retain sets, held-out instruction paraphrases, and four-axis evaluation. The benchmark shows that standard unlearning methods cannot reliably erase the target persona without sacrificing meaningful generation or general utility. We therefore propose PaCE, which compares target and desirable responses to the same questions to locate an internal behavior direction, then trains target-prompt states away from the target mode and toward the matched desirable response. Experiments show that PaCE consistently suppresses target personas with high response quality and useful counterpart behavior, at moderate utility cost. These results establish persona unlearning as a distinct behavior-level editing problem and a practical route toward persistent control of latent LLM response policies.

2026-09-29

arXiv (prépublication)

Preemptive LLM Unlearning against Forbidden Capability Acquisition via Gradient Sealing

Kemou Li

Qizhou Wang

Yue Wang

Fengpeng Li

Masashi Sugiyama

Jiantao Zhou

Open-weight LLMs are released not only as fixed products but also as substrates for downstream fine-tuning. This openness, however, creates … (voir plus)legal and ethical risks because users may misuse fine-tuning to instill illicit knowledge or enable hostile operations. Model providers therefore need apre-release defense against such acquisition, motivating the problem of preemptive unlearning. Unlike retrospective unlearning, which removes capabilities already present in a fixed model, preemptive unlearning seeks to prevent their acquisition under unseen attack data and future fine-tuning procedures. Despite its practical importance, this setting remains largely unexplored, presents distinct challenges, and is therefore the central focus of our work. We first verify that existing retrospective methods provide insufficient pre-release protection. Even when forbidden capabilities are suppressed in current outputs, forbidden-domain data can still induce gradients through internal pathways, enabling later acquisition. Motivated by this finding, we propose a gradient-sealing principle that blocks these pathways by pushing relevant pre-activations into the negative region, where ReLU-family activations exhibit zero or near-zero derivatives. Experiments across multiple LLM families demonstrate our stronger resistance to downstream acquisition than retrospective baselines, validating gradient sealing as an effective mechanism for pre-release protection.

2026-09-29

arXiv (prépublication)

LLJ Cards: Best practices for the Use of LLMs as Judges

Khaoula Chehbouni

Melina Medjdoub

Jackie Chi Kit Cheung

In recent years, large language models (LLMs) have emerged as a popular alternative for evaluation. Often referred to as LLMs as judges (LLJ… (voir plus)s), these systems have been widely adopted by researchers and practitioners across a broad range of measurement tasks, driven by their strong performance, scalability, and cost-effectiveness relative to human judgment. However, a growing body of work has shown that the use of LLJs raise concerns about their validity and reliability as evaluators. Existing efforts to address these challenges have largely focused on developing bias-mitigation techniques and refining prompting strategies. While these approaches represent an important step forward, they primarily offer technical fixes and leave a more fundamental challenge unaddressed: the lack of standardized, transparent, and reproducible evaluation practices. In this paper, we introduce LLJ Cards, a framework that synthesizes best practices from measurement theory, natural language generation, and machine learning literature into practical guidelines for LLJ-based evaluations. While LLJs offer a promising path toward scalable evaluation, their effective use requires grounding in rigorous evaluation principles to ensure validity, reliability, and reproducibility. LLJ Cards addresses this need by providing a structured framework for applying these principles in the design and reporting of automated evaluations.

2026-09-20

arXiv (prépublication)

TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models

Alireza Farashah

Text-to-image diffusion models can be misused to generate harmful content through adversarial or paraphrased prompts that bypass built-in sa… (voir plus)fety mechanisms. Existing concept erasure methods often suffer from limited robustness against adversarial prompts, degradation of benign generation quality, or reliance on inference-time interventions that introduce persistent computational overhead. To address these limitations, we formulate concept erasure as a domain alignment problem in the text representation space. We propose a lightweight Text Encoder Alignment framework (TEA) that fine-tunes only the text encoder while keeping the generative backbone fully frozen. Given concept--anchor prompt pairs, our method trains a discriminator to distinguish token-level representations of concept-containing prompts from those of safe anchor prompts, while updating the text encoder to make these representations indistinguishable. TEA introduces zero inference-time overhead and requires only a small number of fine-tuning steps, making it highly efficient to deploy at scale. Despite this efficiency, TEA achieves state-of-the-art erasure robustness against black-box and white-box adversarial attacks on Stable Diffusion v1.4, while preserving generation quality on benign prompts. Furthermore, TEA is model-agnostic and achieves the lowest attack success rate on Stable Diffusion v3.5, extending concept erasure to a Rectified Flow Transformer architecture with T5 conditioning where prior methods remain largely unexplored. Code is available at \href{https://github.com/alirezafarashah/TEA.git}{https://github.com/alirezafarashah/TEA.git}

2026-08-14

arXiv (prépublication)

IP Protection in the Era of Visual Generative AI: A Survey

Alireza Dehghanpour Farashah

Shunchang Liu

Qian Yang

Han Yu

Cao Yang

Chaochao Chen

Yuping Yan

Yaochu Jin

Lingjuan Lyu

The rapid evolution of visual generative AI has introduced a wide range of intellectual property risks, spanning the unauthorized learning, … (voir plus)reproduction, extraction, misuse, and redistribution of protected data and model assets. To address these risks, a growing body of technical defenses has been proposed. However, existing surveys typically organize this literature by lifecycle stage or technical mechanism, which can obscure the protective intent of different methods. This survey presents a two-dimensional taxonomy for IP protection in visual generative models. The primary axis is a Control Logic View, which classifies methods into Information Exposure Control, Generative Behavior Constraint, and Attribution&Accountability according to the risk variable they regulate. The secondary axis distinguishes Data IP from Model IP as cross-cutting asset dimensions. Under this framework, we systematically review protection methods, align evaluation protocols with protection objectives, and discuss open challenges including proactive model-level safeguards, standardized evaluation, robustness against adaptive attacks, and explainable evidence. This survey aims to offer a principled, systematic, and easy-to-follow overview for both new and experienced researchers in visual generative AI IP protection.

2026-08-12

arXiv (prépublication)

Reasoning with Preference Constraints: A Benchmark for Language Models in Many-to-One Matching Markets

2026-08-01

Transactions on Machine Learning Research (accepté)

openreview.net

Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systems

Large Language Models (LLMs)-powered multi-agent systems are increasingly deployed in mixed-motive environments, where agents operate under … (voir plus)asymmetric information and strategic deception due to conflicting or hidden objectives. In these settings, misalignment with collective goals becomes a central concern. We propose a novel framework for evaluating objective misalignment using the social deduction game Werewolf, modifying the objective of a single agent while preserving its assigned role. Across LLMs from four different model families and sizes, four player roles, and three objective formulations, we introduce a dual analysis of the agents' internal reasoning and their public cheap-talk behavior (i.e costless, non-binding communication that does not directly affect the agents' utilities), complemented by an analysis of game outcomes. Our results show that objective misalignment undermines outcomes in inherently adversarial environments, an effect exacerbated by asymmetric information and specialized roles. While compromised agents consistently develop distinct objective-dependent reasoning strategies, these adaptations remain largely invisible in their public behavior. More broadly, our findings suggest that even subtle objective misalignment can profoundly affect collective decision-making, highlighting the need for effective mitigation strategies for LLM-based multi-agent systems.

2026-07-27

arXiv (prépublication)

Can LLMs Cook Jamaican Couscous? A Study of Cultural Novelty in Recipe Generation

Romain Rampa

2026-06-24

ACM Conference on Fairness, Accountability, and Transparency (publié)

Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior

Ali Khalil

Aly M. Kassem

Mohamed Abdelrazek

Santu Rana

We investigate whether harmful chain-of-thought (CoT) traces from compromised language models can transfer unsafe behaviour and be distilled… (voir plus) into reusable jailbreak attacks. Using an emergent-misalignment organism and a refusal-ablated jailbroken organism, we transplant harmful CoTs into

2026-06-17

arXiv (prépublication)