Golnoosh Farnadi

Biographie

Golnoosh Farnadi est professeure associée à l'École d'informatique de l'Université McGill et professeure associée à l'Université de Montréal. Elle est membre académique principal à Mila - Institut québécois d'intelligence artificielle et est titulaire d'une chaire CIFAR d'intelligence artificielle au Canada.

Mme Farnadi a fondé le laboratoire EQUAL à Mila / Université McGill, dont elle est l'une des principales chercheuses. Le laboratoire EQUAL (EQuity & EQuality Using AI and Learning algorithms) est un laboratoire de recherche de pointe dédié à l'avancement des domaines de l'équité algorithmique et de l'IA responsable.

Étudiants actuels

Mina Arzaghi

Doctorat - HEC

mina.arzaghi@mila.quebec

florian.carichon@mila.quebec

Florian Carichon

Postdoctorat - McGill

Khaoula Chehbouni

Doctorat - McGill

Co-superviseur⋅e :

Doctorat - McGill

Co-superviseur⋅e :

Pablo Piantanida

clea.chataigner@mila.quebec

Alireza Dehghanpour Farashah

Doctorat - McGill

Co-superviseur⋅e :

Negar Rostamzadeh

alireza.farashah@mila.quebec

marylou.fauchard@mila.quebec

Marylou Fauchard

Maîtrise recherche - UdeM

Superviseur⋅e principal⋅e :

Margarida Carvalho

prakhar.ganesh@mila.quebec

Doctorat - McGill

aditi.khandelwal@mila.quebec

Aly Kassem

Collaborateur·rice de recherche - UWindsor

aly.kassem@mila.quebec

Doctorat - McGill

Co-superviseur⋅e :

Siva Reddy

soumya.sharma@mila.quebec

Soumya Sharma

Doctorat - McGill

Co-superviseur⋅e :

Adriana Romero Soriano

william.st-arnaud@mila.quebec

Zhuan Shi

Postdoctorat - McGill

zhuan.shi@mila.quebec

William St-Arnaud

Doctorat - UdeM

Co-superviseur⋅e :

Margarida Carvalho

Emma Tomiuk Tomiuk

Maîtrise recherche - McGill

emma.tomiuk@mila.quebec

Publications

Algorithmic Fairness Across Alignment Procedures and Agentic Systems

Zeyu Tang

Awa Dieng

Miriam Rateike

Jamelle Watson-Daniels

Jessica Schrouff

Sanmi Koyejo

AI has transitioned from predictive models to interactive, autonomous agents capable of reasoning, planning, and executing complex goals. As… (voir plus) the systems increasingly influence social, economic, and scientific decisions, they determine whose interests are represented and whose opportunities are constrained. Ensuring fairness, therefore, is no longer an ethical preference but a practical imperative. As the fairness challenges are fundamentally transformed by advanced AI systems, traditional algorithmic fairness frameworks developed primarily for prediction and/or prediction-based decision-making no longer suffice. This workshop, _Algorithmic Fairness Across Alignment Procedures and Agentic Systems_ (AFAA), emerges at this pivotal moment as a timely forum for rethinking fairness in AI alignment processes and agentic system development. By examining fairness across alignment procedures and agentic systems, this workshop creates a crucial platform for bridging the gap between rapid technical advances in model capabilities and the equally important advances needed in frameworks of algorithmic fairness to govern these powerful systems.

2025-12-31

Workshop Proposals @ International Conference on Learning Representations (publié)

Understanding the role of depth in the neural tangent kernel for overparameterized neural networks

William St-Arnaud

Margarida Carvalho

2025-11-09

ArXiv (prépublication)

arxiv.org

Fairness in Federated Learning: Fairness for Whom?

Afaf Taïk

Khaoula Chehbouni

Fairness in federated learning has emerged as a rapidly growing area of research, with numerous works proposing formal definitions and algor… (voir plus)ithmic interventions. Yet, despite this technical progress, fairness in FL is often defined and evaluated in ways that abstract away from the sociotechnical contexts in which these systems are deployed. In this paper, we argue that existing approaches tend to optimize narrow system level metrics, such as performance parity or contribution-based rewards, while overlooking how harms arise throughout the FL lifecycle and how they impact diverse stakeholders. We support this claim through a critical analysis of the literature, based on a systematic annotation of papers for their fairness definitions, design decisions, evaluation practices, and motivating use cases. Our analysis reveals five recurring pitfalls: 1) fairness framed solely through the lens of server client architecture, 2) a mismatch between simulations and motivating use-cases and contexts, 3) definitions that conflate protecting the system with protecting its users, 4) interventions that target isolated stages of the lifecycle while neglecting upstream and downstream effects, 5) and a lack of multi-stakeholder alignment where multiple fairness definitions can be relevant at once. Building on these insights, we propose a harm centered framework that links fairness definitions to concrete risks and stakeholder vulnerabilities. We conclude with recommendations for more holistic, context-aware, and accountable fairness research in FL.

2025-10-14

Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (publié)

arxiv.org

Mitigating Disparate Impact of Differential Privacy in Federated Learning through Robust Clustering

Saber Malekmohammadi

Afaf Taïk

Federated Learning (FL) is a decentralized machine learning (ML) approach that keeps data localized and often incorporates Differential Priv… (voir plus)acy (DP) to enhance privacy guarantees. Similar to previous work on DP in ML, we observed that differentially private federated learning (DPFL) introduces performance disparities, particularly affecting minority groups. Recent work has attempted to address performance fairness in vanilla FL through clustering, but this method remains sensitive and prone to errors, which are further exacerbated by the DP noise in DPFL. To fill this gap, in this paper, we propose a novel clustered DPFL algorithm designed to effectively identify clients' clusters in highly heterogeneous settings while maintaining high accuracy with DP guarantees. To this end, we propose to cluster clients based on both their model updates and training loss values. Our proposed approach also addresses the server's uncertainties in clustering clients' model updates by employing larger batch sizes along with Gaussian Mixture Model (GMM) to alleviate the impact of noise and potential clustering errors, especially in privacy-sensitive scenarios. We provide theoretical analysis of the effectiveness of our proposed approach. We also extensively evaluate our approach across diverse data distributions and privacy budgets and show its effectiveness in mitigating the disparate impact of DP in FL settings with a small computational cost.

2025-10-06

TMLR (accepté)

Neither Valid Nor Reliable? Investigating the Use of LLMs as Judges

Khaoula Chehbouni

Mohammed Haddou

Jackie CK Cheung

2025-09-24

NeurIPS.cc/2025/Position_Paper_Track (accepté)

Alireza Dehghanpour Farashah

Localized-Attention-Guided Concept Erasure for Text-to-Image Diffusion Models

Zhuan Shi

Rik de Vries

2025-09-23

NeurIPS.cc/2025/Workshop/GenProCC (publié)

Alireza Dehghanpour Farashah

Intrinsic Meets Extrinsic Fairness: Assessing the Downstream Impact of Bias Mitigation in Large Language Models

Mina Arzaghi

Florian Carichon

Large Language Models (LLMs) are increasingly deployed in sensitive domains such as finance, where intrinsic representational biases can pro… (voir plus)pagate into extrinsic harms in downstream tasks. High-stakes applications such as credit scoring are especially vulnerable, as biased model behavior can reinforce existing inequities and result in harmful disparities across demographic groups \cite{blodgett2020language}. While prior research has questioned whether intrinsic bias truly translates into extrinsic unfairness \cite{goldfarb2020intrinsic}, this connection remains poorly understood. To address this gap, we propose a four-stage evaluation framework that systematically examines the relationship between intrinsic and extrinsic fairness. In Stage 1, we establish a baseline by training models such as logistic regression, LLM embeddings, and fine-tuned classifiers without any mitigation strategy, providing reference points for fairness and accuracy. In Stage 2, we evaluate task-level mitigation through Counterfactual Data Augmentation (CDA) \cite{gallegos2024bias}, which balances gender representation by generating counterfactual training instances, allowing us to assess improvements in extrinsic fairness. In Stage 3, we adapt concept unlearning \cite{dige2024mitigating} as an intrinsic bias mitigation method, encouraging LLMs to forget socioeconomic stereotypes while preserving fluency and predictive utility, and we evaluate how this intervention impacts downstream fairness. Finally, in Stage 4, we combine CDA with unlearning to test whether dual mitigation further enhances fairness. We conduct experiments on three datasets (Adult Census Income, ACS Employment, and German Credit) using instruction-tuned LLMs (LLaMA-3.1, Phi-3, and Gemma-2) in both frozen embedding and fine-tuned classifier settings, evaluating performance with predictive accuracy and group fairness metrics, including Demographic Parity, Accuracy Parity, and Equality of Odds. Our experiments demonstrate that intrinsic bias mitigation through unlearning is highly effective; in Phi-3, for instance, it reduces gender socioeconomic stereotype gaps by 94.9\% while maintaining language fluency. In downstream tasks, unlearning consistently improves group fairness metrics while preserving predictive accuracy, whereas CDA primarily enhances demographic parity but can introduce accuracy trade-offs. For instance, on the ACS Employment dataset, unlearned Gemma-2 improved Accuracy Parity from 0.199 to 0.104 (48\% gain), and combining CDA with unlearning on Llama-3.1 reduced Demographic Parity from 0.080 to 0.014 (82\% gain). On the Adult dataset, all three models maintained accuracy above 0.82 while showing reduced fairness gaps, and on German Credit, unlearning consistently outperformed CDA by improving group fairness metrics without sacrificing predictive performance. Overall, CDA and unlearning exhibit complementary effects, with their combination yielding the strongest fairness improvements across models and datasets. This work contributes to bias mitigation and fairness in LLMs in two ways. First, we adapt concept unlearning to mitigate socioeconomic stereotyping, showing that intrinsic bias reduction improves both representational and downstream fairness. Second, we introduce a unified evaluation framework that links intrinsic and extrinsic fairness, enabling systematic comparison of mitigation strategies. The framework is flexible, applying to both fine-tuned and frozen LLMs, and offers actionable guidance for deploying fairer models in finance and other high-stakes domains.

2025-09-21

NeurIPS.cc/2025/Workshop/WiML (publié)

Reasoning with Preference Constraints: A Benchmark for Language Models in Many-to-One Matching Markets

2025-09-21

WiML @ Neural Information Processing Systems (publié)

Say It Another Way: Auditing LLMs with a User-Grounded Automated Paraphrasing Framework

Cléa Chataigner

Rebecca Ma

Afaf Taïk

Elliot Creager

2025-09-21

NeurIPS.cc/2025/Workshop/WiML (publié)

Towards Democratizing LLMs: Investigating Multilingual Mixture-of-Experts Models

2025-09-21

NeurIPS.cc/2025/Workshop/WiML (publié)

Algorithmic Fairness Through the Lens of Metrics and Evaluation (AFME) 2024

Miriam Rateike

Awa Dieng

Jamelle Watson-Daniels

Ferdinando Fioretto

2025-04-22

Proceedings of the Algorithmic Fairness Through the Lens of Metrics and Evaluation (publié)

proceedings.mlr.press

Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in ML

Usman Gohar

Lu Cheng

With fairness concerns gaining significant attention in Machine Learning (ML), several bias mitigation techniques have been proposed, often … (voir plus)compared against each other to find the best method. These benchmarking efforts tend to use a common setup for evaluation under the assumption that providing a uniform environment ensures a fair comparison. However, bias mitigation techniques are sensitive to hyperparameter choices, random seeds, feature selection, etc., meaning that comparison on just one setting can unfairly favour certain algorithms. In this work, we show significant variance in fairness achieved by several algorithms and the influence of the learning pipeline on fairness scores. We highlight that most bias mitigation techniques can achieve comparable performance, given the freedom to perform hyperparameter optimization, suggesting that the choice of the evaluation parameters-rather than the mitigation technique itself-can sometimes create the perceived superiority of one method over another. We hope our work encourages future research on how various choices in the lifecycle of developing an algorithm impact fairness, and trends that guide the selection of appropriate algorithms.

2025-04-22

Algorithmic Fairness Through the Lens of Metrics and Evaluation @ International Conference on Machine Learning (publié)