Su Lin Blodgett

Lisa Egede

Myra Cheng

2026-06-22

ACM Conference on Fairness, Accountability, and Transparency (published)

On Defining Erasure Harms for NLP

Yu Lu Liu

Arnav Goel

Jackie Chi Kit Cheung

Alexandra Olteanu

Ziang Xiao

The deployment of NLP systems has raised concerns about harms they might produce, including representational harms. Recent literature has be… (see more)gun to conceptualize and measure one such harm, the harm of erasure. Nevertheless, the field lacks a clear and cohesive conceptual foundation for identifying and measuring erasure. Existing conceptualizations of erasure are often broad -- making it difficult to identify what is needed to establish and measure erasure -- or else specific to particular settings -- facilitating measurement for those settings but potentially challenging to adapt to other settings. To address this gap, we develop and propose a structured definition of erasure that clarifies what components are necessary for establishing whether erasure has occurred, which practitioners need to explicitly articulate and operationalize in order to measure erasure.

2026-06-13

arXiv (preprint)

Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning

Shravika Mittal

Q. Vera Liao

Generative AI (GenAI) tools offer increasing opportunities for augmenting human cognitive tasks. Among these tasks, information seeking is b… (see more)eing rapidly reshaped by GenAI tools, with potentially profound implications for learning and knowledge acquisition. To investigate these implications, we conducted a between-subjects field experiment in which participants pursued informal learning by seeking information through either ChatGPT or Google Search over a span of 8 days. Using a daily diary protocol, we gathered in-situ data on their information-seeking processes. Our findings show that participants in the ChatGPT group experienced diminished agency in their information-seeking processes, as they offloaded much of the information selection to AI, and consequently experienced greater meta-cognitive load arising from this reduced sense of control. We further highlight two sources of distortion in information access when using ChatGPT: biases in ChatGPT outputs, particularly towards providing solution-oriented artifacts over principled knowledge; and systematic shifts in users' information-seeking behaviors, whereby the conversational and socially-oriented interaction paradigm of current GenAI tools may inadvertently reduce exploration of the broader knowledge space. As a result, on average, participants in the ChatGPT group had worse learning outcomes than those using Google, especially for higher-order critical learning. Our work suggests inherent tensions between offloading information seeking to AI and meaningful learning, and provides broader implications for understanding AI's risks to human cognition.

2026-06-09

arXiv (preprint)

Illusions of the Gold Standard: A Large-scale Analysis of Human Evaluation Protocols for Long-form Text Generation

Katelyn Xiaoying Mei

Yi-Li Hsu

Minjoon Choi

Zongwan Cao

Chenjun Xu

Bingbing Wen

Lucy Lu Wang

Human evaluation plays a critical role in assessing the quality of generated text. However, the reliability and reproducibility of these eva… (see more)luations depend on transparent and well-documented protocols -- details that are frequently missing in current practice. In this work, we conduct a large-scale analysis of human evaluation protocols for evaluating long-form generation tasks in *CL conference publications from 2023--2025, including a full manual review of 284 papers and LLM-assisted analysis for another 1.8k+ papers. We define a set of 20 reportable criteria related to reproducibility of human evaluation studies, and apply these criteria to systematically examine reporting norms and practices within the community. We find widespread under-reporting of important aspects of human evaluation study design, leading to ambiguity about what was measured and how, who contributed judgments, and how judgments should be interpreted. Based on these findings, we outline actionable recommendations to support more transparent and reproducible reporting in future research. Our analysis code and annotated dataset can be found at: https://github.com/larchlab/Illusions-of-the-Gold-Standard

2026-06-05

arXiv (preprint)

From Use to Oversight: How Mental Models Influence User Behavior and Output in AI Writing Assistants

Q. Vera Liao

AI-based writing assistants are ubiquitous, yet little is known about how users' mental models shape their use. We examine two types of ment… (see more)al models -- functional or related to what the system does, and structural or related to how the system works -- and how they affect control behavior -- how users request, accept, or edit AI suggestions as they write -- and writing outcomes. We primed participants (

2026-04-05

arXiv (preprint)

“I Was a Young AI”: On Probing the Effectiveness of Intervening on Anthropomorphic AI System Outputs

Myra Cheng

Alexandra Olteanu

We see growing concerns about how the increasingly pervasive deployment of AI systems whose outputs appear human-like might impact people.Th… (see more)ese concerns have already motivated work both examining what makes such outputs appear human-like, as well as developing interventions to help reduce perceptions of human-likeness or mitigate adverse impacts.In this paper, we report on an exploratory crowd study we designed to examine challenges for assessing the effectiveness of interventions, including whether interventions intended to minimize perceptions of human-likeness also mitigate adverse impacts.We find variations both in what kinds of outputs different participants deem more human-like, as well as in their preferences for human-like outputs.Even when participants seem to prefer the outputs they deem more human-like, many of them also recognize that such outputs can have adverse impacts.Drawing on these results and prior work, we discuss challenges to and considerations for assessing the effectiveness of interventions.

2025-12-31

Beyond Alignment: Transdisciplinary Conversations on Human-AI Futures @ Neural Information Processing Systems (published)

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

Hanna Wallach

Meera Desai

A. Feder Cooper

Angelina Wang

Chad Atalla

Solon Barocas

Alexandra Chouldechova

Emily Corvi

P. A. Dow

Jean Garcia-Gathright

A.R. Olteanu

Nicholas Pangakis

Stefanie Reed

Emily Sheng

Dan Vann

Jennifer Wortman Vaughan

Matthew Vogel

Hannah Washington

Abigail Z. Jacobs

The measurement tasks involved in evaluating generative AI (GenAI) systems are especially difficult, leading to what has been described as"a… (see more) tangle of sloppy tests [and] apples-to-oranges comparisons"(Roose, 2024). In this position paper, we argue that the ML community would benefit from learning from and drawing on the social sciences when developing and using measurement instruments for evaluating GenAI systems. Specifically, our position is that evaluating GenAI systems is a social science measurement challenge. We present a four-level framework, grounded in measurement theory from the social sciences, for measuring concepts related to the capabilities, behaviors, and impacts of GenAI. This framework has two important implications for designing and evaluating evaluations: First, it can broaden the expertise involved in evaluating GenAI systems by enabling stakeholders with different perspectives to participate in conceptual debates. Second, it brings rigor to both conceptual and operational debates by offering a set of lenses for interrogating the validity of measurement instruments and their resulting measurements.

2025-10-05

Proceedings of the 42nd International Conference on Machine Learning (published)

proceedings.mlr.press

Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor

A.R. Olteanu

Agathe Balayn

Angelina Wang

Fernando Diaz

Flavio Calmon

Margaret Mitchell

Michael Ekstrand

Reuben Binns

Solon Barocas

In AI research and practice, rigor remains largely understood in terms of methodological rigor -- such as whether mathematical, statistical,… (see more) or computational methods are correctly applied. We argue that this narrow conception of rigor has contributed to the concerns raised by the responsible AI community, including overblown claims about AI capabilities. Our position is that a broader conception of what rigorous AI research and practice should entail is needed. We believe such a conception -- in addition to a more expansive understanding of (1) methodological rigor -- should include aspects related to (2) what background knowledge informs what to work on (epistemic rigor); (3) how disciplinary, community, or personal norms, standards, or beliefs influence the work (normative rigor); (4) how clearly articulated the theoretical constructs under use are (conceptual rigor); (5) what is reported and how (reporting rigor); and (6) how well-supported the inferences from existing evidence are (interpretative rigor). In doing so, we also aim to provide useful language and a framework for much-needed dialogue about the AI community's work by researchers, policymakers, journalists, and other stakeholders.

2025-09-24

NeurIPS.cc/2025/Position_Paper_Track (accepted)

openreview.net

"It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Models

Angel Hsing-Chi Hwang

Q. Vera Liao

A.R. Olteanu

Adam Trischler

2025-05-01

Proceedings of the ACM on Human-Computer Interaction (published)

AI Automatons: AI Systems Intended to Imitate Humans

A.R. Olteanu

Solon Barocas

Lisa Egede

Alicia DeVrio

Myra Cheng

There is a growing proliferation of AI systems designed to mimic people's behavior, work, abilities, likenesses, or humanness -- systems we … (see more)dub AI automatons. Individuals, groups, or generic humans are being simulated to produce creative work in their styles, to respond to surveys in their places, to probe how they would use a new system before deployment, to provide users with assistance and companionship, and to anticipate their possible future behavior and interactions with others, just to name a few applications. The research, design, deployment, and availability of such AI systems have, however, also prompted growing concerns about a wide range of possible legal, ethical, and other social impacts. To both 1) facilitate productive discussions about whether, when, and how to design and deploy such systems, and 2) chart the current landscape of existing and prospective AI automatons, we need to tease apart determinant design axes and considerations that can aid our understanding of whether and how various design choices along these axes could mitigate -- or instead exacerbate -- potential adverse impacts that the development and use of AI automatons could give rise to. In this paper, through a synthesis of related literature and extensive examples of existing AI systems intended to mimic humans, we develop a conceptual framework to help foreground key axes of design variations and provide analytical scaffolding to foster greater recognition of the design choices available to developers, as well as the possible ethical implications these choices might have.

2025-03-03

ArXiv (preprint)

A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologies

Alicia DeVrio

Myra Cheng

Lisa Egede

A.R. Olteanu

Recent attention to anthropomorphism -- the attribution of human-like qualities to non-human objects or entities -- of language technologies… (see more) like LLMs has sparked renewed discussions about potential negative impacts of anthropomorphism. To productively discuss the impacts of this anthropomorphism and in what contexts it is appropriate, we need a shared vocabulary for the vast variety of ways that language can be anthropomorphic. In this work, we draw on existing literature and analyze empirical cases of user interactions with language technologies to develop a taxonomy of textual expressions that can contribute to anthropomorphism. We highlight challenges and tensions involved in understanding linguistic anthropomorphism, such as how all language is fundamentally human and how efforts to characterize and shift perceptions of humanness in machines can also dehumanize certain humans. We discuss ways that our taxonomy supports more precise and effective discussions of and decisions about anthropomorphism of language technologies.

2025-02-13

ArXiv (preprint)

Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems

Myra Cheng

Alicia DeVrio

Lisa Egede

A.R. Olteanu

As text generation systems' outputs are increasingly anthropomorphic -- perceived as human-like -- scholars have also raised increasing conc… (see more)erns about how such outputs can lead to harmful outcomes, such as users over-relying or developing emotional dependence on these systems. How to intervene on such system outputs to mitigate anthropomorphic behaviors and their attendant harmful outcomes, however, remains understudied. With this work, we aim to provide empirical and theoretical grounding for developing such interventions. To do so, we compile an inventory of interventions grounded both in prior literature and a crowdsourced study where participants edited system outputs to make them less human-like. Drawing on this inventory, we also develop a conceptual framework to help characterize the landscape of possible interventions, articulate distinctions between different types of interventions, and provide a theoretical basis for evaluating the effectiveness of different interventions.

2024-12-31

ACL (1) (published)