Portrait of Golnoosh Farnadi

Golnoosh Farnadi

Core Academic Member
Canada CIFAR AI Chair
Assistant Professor, McGill University
Adjunct Professor, Université de Montréal, Department of Computer Science and Operations Research
Visiting Faculty Researcher, Google
Research Topics
Deep Learning
Generative Models

Biography

Golnoosh Farnadi is an assistant professor at the School of Computer Science, McGill University, and an adjunct professor at Université de Montréal. She is a core academic member of Mila – Quebec Artificial Intelligence Institute and holds a Canada CIFAR AI Chair.

Farnadi founded and is a principal investigator of the EQUAL lab at Mila / McGill University. The EQUAL lab (EQuity & EQuality Using AI and Learning algorithms) is a cutting-edge research laboratory dedicated to advancing the fields of algorithmic fairness and responsible AI.

Current Students

Postdoctorate - McGill University
PhD - McGill University
Co-supervisor :
Undergraduate - McGill University
Master's Research - Université de Montréal
Principal supervisor :
Collaborating researcher - UWindsor
Undergraduate - McGill University
Master's Research - McGill University
Undergraduate - McGill University
Postdoctorate - McGill University
PhD - Université de Montréal
Co-supervisor :
Master's Research - McGill University

Publications

Auditing is not Evaluating: LLM Audit Requires Dynamic, Contextual, Budget-aware and Reliable Evidence
Frank-LoRA: Federated Rank-Aware LoRA for Fine-Tuning Large Models
Kevin Kuo
Virginia Smith
From Infrastructure to Interface, the AI Value Chain Drives LLM Homogenization
Scaling Laws for Multimodal Data Mixtures
Ayush K. Tarun
Yixuan Xu
Imanol Schlag
Luke Zettlemoyer
Antoine Bosselut
Stochastic Projections and Rank-Aware Learning Rates for Federated LoRA
Kevin Kuo
Virginia Smith
Fine-tuning large-scale foundation models across resource-constrained, distributed clients presents significant computational and communicat… (see more)ion challenges. While merging Low-Rank Adaptation (LoRA) with Federated Learning (FL) enables rank-based flexibility, the interdependence of the adaptation matrices (
Bridging Neural Tangent Kernel Analysis and Reinforcement Learning for Deep Function Approximation
Despite their widespread success across machine learning, the learning dynamics and generalization properties of neural networks remain poor… (see more)ly understood. The neural tangent kernel (NTK) provides an analytical characterization of the learning dynamics of infinitely wide neural networks and serves as a useful proxy for finite-width networks in supervised learning. In this work, we extend the NTK framework to reinforcement learning (RL) by introducing a temporal-difference (TD) flow formulation that accommodates time-varying mini-batches and TD targets, yielding a closed-form solution for value function evolution in an overparametrized setting. We also devise an optimization-based state-space exploration method that leverages the closed-form solution, coupled with constraints on the environment, to model the set of feasible states. On a deterministic grid environment, we validate that the proposed RL-based NTK formulation is indeed learning efficiently and adapting to the distribution shift in the training data distribution, and we demonstrate that the resulting exploration strategy improves sample efficiency. To our knowledge, this is the first work to extend NTK theory to characterize the learning dynamics of RL training with time-varying mini-batches and TD updates.
Privacy in Context Is Being Overlooked in the Age of General-Purpose LLMs
Yash More
Marco Romanelli
Ferdinando Fioretto
As large language models (LLMs) continue to grow in scope and capability, privacy concerns are on the rise. Since users increasingly interac… (see more)t with a single general-purpose system across many distinct contexts, there is a natural tendency to pursue a one-size-fits-all solution to privacy. Unfortunately, this overlooks a crucial insight from decades of privacy research: privacy is inherently contextual. In this paper, we argue that existing work in prompt privacy is drifting towards attempts to design a unified framework, which is highly misleading and can foster a false sense of security. Furthermore, we argue that even future work in this direction needs to wrestle with fundamental tensions across various desirable properties of prompt privacy. Thus, such continued attempts towards a unified framework need to be reconsidered. We recommend three critical paths forward: application-specific deployment of LLMs, context-specific evaluation of privacy frameworks, and better privacy literacy.
Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs
Large Language Models (LLMs) are increasingly used in high-stakes decision-making systems, where biased predictions can reinforce social and… (see more) economic disparities. Although prior work has examined intrinsic representational bias and unfair downstream behavior separately, it remains unclear whether mitigating intrinsic bias leads to fairer downstream outcomes. We introduce Fairness-Aware Concept Unlearning (FACU), a model-level mitigation method that adapts concept unlearning to fairness-oriented representation balancing. Unlike suppression-based approaches, FACU explicitly regularizes probability differences between stereotypical and anti-stereotypical associations while preserving predictive performance and language modeling quality. We evaluate FACU across three open-source LLMs, multiple intrinsic bias benchmarks, and three socio-economic classification datasets using both frozen LLM embeddings and LoRA-fine-tuned classifiers. FACU produces statistically significant reductions in intrinsic gender bias that are associated with downstream fairness improvements across most evaluated settings, datasets, models, and fairness metrics, without significantly degrading predictive performance. Combining FACU with extrinsic mitigation methods, particularly counterfactual data augmentation, yields further fairness improvements. These findings suggest that fairness-aware intrinsic mitigation can support fairer LLM-based decision-making and that bias mitigation should be addressed across both model development and downstream deployment stages.
Loose Lips: Understanding Privacy Leakage in Agentic Dialogue
As LLM agents are increasingly deployed in high-stakes enterprise settings, understanding how they handle sensitive information in agentic d… (see more)ialogue has become critical. Privacy leakage in such settings requires reasoning about whether information flows respect the contextual norms under which they originated. We extend CIMemories to CIMEMORIES DIALOGUE, a benchmark of 4,410 two-agent dialogue instances grounded in Contextual Integrity theory. Using this benchmark, we systematically evaluate 7 open-weight models from the Qwen3 and Llama 3.1 families across conversation lengths of 1 and 6 turns under three prompting conditions. We find that task capability and privacy leakage are fundamentally coupled: more capable models are systematically leakier, with no model in our evaluation able to decouple the two. Thinking models consistently leak more than their non-thinking counterparts, a pattern we attribute to the extended reasoning surface created by chain-of-thought. Leakage compounds with conversation length, rising up to 3x from 1 to 6 turns. Our results suggest that the coupling between task capability and privacy leakage is not an artifact of misconfiguration but a structural feature of current agentic systems.
Beyond Fair Answers: Learning Fair and Unlearning Biased Reasoning in LLMs
Large Language Models (LLMs) have demonstrated strong reasoning capabilities on tasks with well-defined solutions, such as mathematical and … (see more)logical reasoning. However, reasoning capability does not necessarily imply unbiased reasoning. In socially ambiguous contexts, where the available evidence is insufficient to support one conclusion over another, models may rely on demographic stereotypes or unsupported social assumptions. Recent work shows that chain-of-thought (CoT) reasoning can contain social stereotypes and that biased reasoning steps are associated with stereotype expression and incorrect predictions. This motivates evaluating fairness in the reasoning process, not only in the final answer. We therefore ask: Can we train LLMs not only to produce fair answers, but to reason fairly? We study this question using a fairness-oriented adaptation of BBQ, where ambiguous cases test whether demographic alternatives receive comparable probability when evidence is insufficient, while disambiguated cases test evidence-grounded accuracy. Following reasoning-trace fine-tuning approaches such as STaR and ReGiFT, we fine-tune instruction-tuned LLMs on reasoning traces generated by stronger models and compare base and fine-tuned models under direct and CoT inference. We evaluate both final decisions and generated reasoning traces, and test generalization on SocioEconomicQA and CrowS-Pairs. Building on this analysis, we propose a reasoning-level fairness framework that jointly learns fair reasoning and unlearns biased reasoning. Evidence-grounded traces form the positive learning set, while traces containing unsupported demographic or stereotype-based inferences form the forget set. Our objective combines supervised fine-tuning on fair traces, a negative objective on biased traces, and KL regularization on unrelated reasoning data to preserve general reasoning ability. We evaluate whether this process improves selection of the fair answer and, more importantly, reduces the probability gap between gendered options when the context provides no evidence favoring either one. We separately measure accuracy in disambiguated contexts to ensure that the model still uses available evidence correctly. At the reasoning level, we evaluate stereotype-dependent inferences and the relative likelihood of fair versus biased reasoning traces. We compare against ReGiFT, standard CoT, and instruction-based reasoning baselines, and evaluate generalization across additional bias benchmarks while monitoring general reasoning performance for capability loss. This work moves fairness intervention from the final decision to the reasoning process that produces it. By combining fair-reasoning supervision, biased-reasoning unlearning, and capability preservation, we investigate whether LLMs can learn not only to give fair answers, but to reason without relying on unsupported demographic stereotypes.
LLM Persona Unlearning
Kemou Li
Qizhou Wang
Fengpeng Li
Jiantao Zhou
Pre-training equips large language models (LLMs) with a broad repertoire of behavioral patterns associated with roles, styles, values, and g… (see more)oals. Post-training teaches conditional enactment and makes a helpful Assistant the default, but it does not erase alternative modes from the weights; explicit prompts can therefore elicit personas that repeatedly shape judgment, language, and action. In open-weight settings, runtime controls can be removed, motivating persona unlearning: a weight-level edit that makes a designated persona difficult to elicit and enact on unseen contexts. We introduce PersonaUnlearnBench, a model-specific paired benchmark spanning six LLMs from three families and five personas, with aligned forget/retain sets, held-out instruction paraphrases, and four-axis evaluation. The benchmark shows that standard unlearning methods cannot reliably erase the target persona without sacrificing meaningful generation or general utility. We therefore propose PaCE, which compares target and desirable responses to the same questions to locate an internal behavior direction, then trains target-prompt states away from the target mode and toward the matched desirable response. Experiments show that PaCE consistently suppresses target personas with high response quality and useful counterpart behavior, at moderate utility cost. These results establish persona unlearning as a distinct behavior-level editing problem and a practical route toward persistent control of latent LLM response policies.
Preemptive LLM Unlearning against Forbidden Capability Acquisition via Gradient Sealing
Kemou Li
Qizhou Wang
Yue Wang
Fengpeng Li
Masashi Sugiyama
Jiantao Zhou
Open-weight LLMs are released not only as fixed products but also as substrates for downstream fine-tuning. This openness, however, creates … (see more)legal and ethical risks because users may misuse fine-tuning to instill illicit knowledge or enable hostile operations. Model providers therefore need apre-release defense against such acquisition, motivating the problem of preemptive unlearning. Unlike retrospective unlearning, which removes capabilities already present in a fixed model, preemptive unlearning seeks to prevent their acquisition under unseen attack data and future fine-tuning procedures. Despite its practical importance, this setting remains largely unexplored, presents distinct challenges, and is therefore the central focus of our work. We first verify that existing retrospective methods provide insufficient pre-release protection. Even when forbidden capabilities are suppressed in current outputs, forbidden-domain data can still induce gradients through internal pathways, enabling later acquisition. Motivated by this finding, we propose a gradient-sealing principle that blocks these pathways by pushing relevant pre-activations into the negative region, where ReLU-family activations exhibit zero or near-zero derivatives. Experiments across multiple LLM families demonstrate our stronger resistance to downstream acquisition than retrospective baselines, validating gradient sealing as an effective mechanism for pre-release protection.