Golnoosh Farnadi

Alireza Dehghanpour Farashah

Biography

Golnoosh Farnadi is an assistant professor at the School of Computer Science, McGill University, and an adjunct professor at Université de Montréal. She is a core academic member of Mila – Quebec Artificial Intelligence Institute and holds a Canada CIFAR AI Chair.

Farnadi founded and is a principal investigator of the EQUAL lab at Mila / McGill University. The EQUAL lab (EQuity & EQuality Using AI and Learning algorithms) is a cutting-edge research laboratory dedicated to advancing the fields of algorithmic fairness and responsible AI.

Current Students

PhD - HEC Montréal

Postdoctorate - McGill University

Khaoula Chehbouni

PhD - McGill University

Co-supervisor :

PhD - McGill University

Co-supervisor :

Pablo Piantanida

PhD - McGill University

Co-supervisor :

Master's Research - Université de Montréal

Principal supervisor :

Margarida Carvalho

Prakhar Ganesh

PhD - McGill University

Website

Github

Aly Kassem

Collaborating researcher - UWindsor

PhD - McGill University

Co-supervisor :

Siva Reddy

Saber Malekmohammadi

Collaborating researcher - McGill University

Yash Raj More

Collaborating Alumni - Université de Montréal

Edie Pearman

Independent visiting researcher - McGill University university

Ameline Ramesan

Collaborating researcher - McGill University

Github

Soumya Sharma

PhD - McGill University

Co-supervisor :

Adriana Romero Soriano

Website

Github

Zhuan Shi

Postdoctorate - McGill University

William St-Arnaud

PhD - Université de Montréal

Co-supervisor :

Margarida Carvalho

Emma Tomiuk Tomiuk

Master's Research - McGill University

Publications

Low-Rank Adaptation Secretly Imitates Differentially Private SGD

Saber Malekmohammadi

As pre-trained language models grow in size, full fine-tuning their parameters on task adaptation data becomes increasingly impractical. To … (see more)address this challenge, some methods for low-rank adaptation of language models have been proposed, e.g. LoRA, which incorporates trainable low-rank decomposition matrices into only some parameters of the pre-trained model, called adapters. This approach significantly reduces the number of trainable parameters compared to fine-tuning all parameters or adapters. In this work, we look at low-rank adaptation method from the lens of data privacy. We show theoretically that the low-rank adaptation used in LoRA is equivalent to fine-tuning adapters with noisy batch gradients - just like what DPSGD algorithm does. We also quantify the variance of the injected noise as a decreasing function of adaptation rank. By establishing a Berry-Esseen type bound on the total variation distance between the injected noise distribution and a Gaussian noise distribution with the same variance, we show that the dynamics of low-rank adaptation is very close to when DPSGD is performed w.r.t the adapters. Following our theoretical findings and approved by our experimental results, we show that low-rank adaptation provides robustness to membership inference attacks w.r.t the fine-tuning data.

2024-10-09

NeurIPS.cc/2024/Workshop/M3L (poster)

Wasserstein Distributionally Robust Optimization Through the Lens of Structural Causal Models and Individual Fairness

Ahmad-reza Ehyaei

Samira Samadi

In recent years, Wasserstein Distributionally Robust Optimization (DRO) has garnered substantial interest for its efficacy in data-driven de… (see more)cision-making under distributional uncertainty. However, limited research has explored the application of DRO to address individual fairness concerns, particularly when considering causal structures and sensitive attributes in learning problems. To address this gap, we first formulate the DRO problem from causality and individual fairness perspectives. We then present the DRO dual formulation as an efficient tool to convert the DRO problem into a more tractable and computationally efficient form. Next, we characterize the closed form of the approximate worst-case loss quantity as a regularizer, eliminating the max-step in the min-max DRO problem. We further estimate the regularizer in more general cases and explore the relationship between DRO and classical robust optimization. Finally, by removing the assumption of a known structural causal model, we provide finite sample error bounds when designing DRO with empirical distributions and estimated causal structures to ensure efficiency and robust learning.

2024-09-24

NeurIPS.cc/2024/Conference (poster)

What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models

Ahmed Imtiaz Humayun

Candice Schumann

2024-08-14

ArXiv (preprint)

From Representational Harms to Quality-of-Service Harms: A Case Study on Llama 2 Safety Safeguards

Khaoula Chehbouni

Megha Roshan

Emmanuel Ma

Futian Andrew Wei

Afaf Taïk

Jackie CK Cheung

2024-07-31

Findings of the Association for Computational Linguistics ACL 2024 (published)

Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild

Niloofar Mireshghallah

Maria Antoniak

Yash More

Yejin Choi

Measuring personal disclosures made in human-chatbot interactions can provide a better understanding of users' AI literacy and facilitate pr… (see more)ivacy research for large language models (LLMs). We run an extensive, fine-grained analysis on the personal disclosures made by real users to commercial GPT models, investigating the leakage of personally identifiable and sensitive information. To understand the contexts in which users disclose to chatbots, we develop a taxonomy of tasks and sensitive topics, based on qualitative and quantitative analysis of naturally occurring conversations. We discuss these potential privacy harms and observe that: (1) personally identifiable information (PII) appears in unexpected contexts such as in translation or code editing (48% and 16% of the time, respectively) and (2) PII detection alone is insufficient to capture the sensitive topics that are common in human-chatbot interactions, such as detailed sexual preferences or specific drug use habits. We believe that these high disclosure rates are of significant importance for researchers and data curators, and we call for the design of appropriate nudging mechanisms to help users moderate their interactions.

2024-07-09

colmweb.org/COLM/2024/Conference (accepted)

Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities

Mohammad Havaei

Negar Rostamzadeh

2024-07-07

Proceedings of the 41st International Conference on Machine Learning (published)

proceedings.mlr.press

On The Local Geometry of Deep Generative Manifolds

Ahmed Imtiaz Humayun

Candice Schumann

In this paper, we study theoretically inspired local geometric descriptors of the data manifolds approximated by pre-trained generative mode… (see more)ls. The descriptors – local scaling (ψ), local rank (ν), and local complexity (δ) — characterize the uncertainty, dimensionality, and smoothness on the learned manifold, using only the network weights and architecture. We investigate and emphasize their critical role in understanding generative models. Our analysis reveals that the local geometry is intricately linked to the quality and diversity of generated outputs. Additionally, we see that the geometric properties are distinct for out-of-distribution (OOD) inputs as well as for prompts memorized by Stable Diffusion, showing the possible application of our proposed descriptors for downstream detection and assessment of pre-trained generative models.

2024-06-16

ICML.cc/2024/Workshop/GRaM (published)

The Cost of Arbitrariness for Individuals: Examining the Legal and Technical Challenges of Model Multiplicity

Prakhar Ganesh

Ihsan Ibrahim Daldaban

Ignacio Cofone

Model multiplicity, the phenomenon where multiple models achieve similar performance despite different underlying learned functions, introdu… (see more)ces arbitrariness in model selection. While this arbitrariness may seem inconsequential in expectation, its impact on individuals can be severe. This paper explores various individual concerns stemming from multiplicity, including the effects of arbitrariness beyond final predictions, disparate arbitrariness for individuals belonging to protected groups, and the challenges associated with the arbitrariness of a single algorithmic system creating a monopoly across various contexts. It provides both an empirical examination of these concerns and a comprehensive analysis from the legal standpoint, addressing how these issues are perceived in the anti-discrimination law in Canada. We conclude the discussion with technical challenges in the current landscape of model multiplicity to meet legal requirements and the legal gap between current law and the implications of arbitrariness in model selection, highlighting relevant future research directions for both disciplines.

2024-05-27

ArXiv (preprint)

Advancing Cultural Inclusivity: Optimizing Embedding Spaces for Balanced Music Recommendations

Armin Moradi

Nicola Neophytou

2024-05-26

ArXiv (preprint)

Fairness Incentives in Response to Unfair Dynamic Pricing

Jesse Thibodeau

Hadi Nekoei

Afaf Taïk

Janarthanan Rajendran

The use of dynamic pricing by profit-maximizing firms gives rise to demand fairness concerns, measured by discrepancies in consumer groups' … (see more)demand responses to a given pricing strategy. Notably, dynamic pricing may result in buyer distributions unreflective of those of the underlying population, which can be problematic in markets where fair representation is socially desirable. To address this, policy makers might leverage tools such as taxation and subsidy to adapt policy mechanisms dependent upon their social objective. In this paper, we explore the potential for AI methods to assist such intervention strategies. To this end, we design a basic simulated economy, wherein we introduce a dynamic social planner (SP) to generate corporate taxation schedules geared to incentivizing firms towards adopting fair pricing behaviours, and to use the collected tax budget to subsidize consumption among underrepresented groups. To cover a range of possible policy scenarios, we formulate our social planner's learning problem as a multi-armed bandit, a contextual bandit and finally as a full reinforcement learning (RL) problem, evaluating welfare outcomes from each case. To alleviate the difficulty in retaining meaningful tax rates that apply to less frequently occurring brackets, we introduce FairReplayBuffer, which ensures that our RL agent samples experiences uniformly across a discretized fairness space. We find that, upon deploying a learned tax and redistribution policy, social welfare improves on that of the fairness-agnostic baseline, and approaches that of the analytically optimal fairness-aware baseline for the multi-armed and contextual bandit settings, and surpassing it by 13.19% in the full RL setting.

2024-04-21

ArXiv (preprint)

Learning to Build Solutions in Stochastic Matching Problems Using Flows (Student Abstract)

William St-Arnaud

Margarida Carvalho

Generative Flow Networks, known as GFlowNets, have been introduced in recent times, presenting an exciting possibility for neural networks t… (see more)o model distributions across various data structures. In this paper, we broaden their applicability to encompass scenarios where the data structures are optimal solutions of a combinatorial problem. Concretely, we propose the use of GFlowNets to learn the distribution of optimal solutions for kidney exchange problems (KEPs), a generalized form of matching problems involving cycles.

2024-03-23

Proceedings of the AAAI Conference on Artificial Intelligence (published)

Promoting Fair Vaccination Strategies Through Influence Maximization: A Case Study on COVID-19 Spread

Nicola Neophytou

Afaf Taïk

The aftermath of the Covid-19 pandemic saw more severe outcomes for racial minority groups and economically-deprived communities. Such dispa… (see more)rities can be explained by several factors, including unequal access to healthcare, as well as the inability of low income groups to reduce their mobility due to work or social obligations. Moreover, senior citizens were found to be more susceptible to severe symptoms, largely due to age-related health reasons. Adapting vaccine distribution strategies to consider a range of demographics is therefore essential to address these disparities. In this study, we propose a novel approach that utilizes influence maximization (IM) on mobility networks to develop vaccination strategies which incorporate demographic fairness. By considering factors such as race, social status, age, and associated risk factors, we aim to optimize vaccine distribution to achieve various fairness definitions for one or more protected attributes at a time. Through extensive experiments conducted on Covid-19 spread in three major metropolitan areas across the United States, we demonstrate the effectiveness of our proposed approach in reducing disease transmission and promoting fairness in vaccination distribution.

2024-03-23

Proceedings of the AAAI Conference on Artificial Intelligence (published)