Publications

Multivariate Conformal Selection
Tian Bai
Yue Zhao
Xiang Yu
Archer Yang
Selecting high-quality candidates from large datasets is critical in applications such as drug discovery, precision medicine, and alignment … (see more)of large language models (LLMs). While Conformal Selection (CS) provides rigorous uncertainty quantification, it is limited to univariate responses and scalar criteria. To address this, we propose Multivariate Conformal Selection (mCS), a generalization of CS designed for multivariate response settings. Our method introduces regional monotonicity and employs multivariate nonconformity scores to construct conformal
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press
Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning
Guozheng Ma
Lu Li
Zilin Wang
Li Shen
Pierre-Luc Bacon
Dacheng Tao
Effectively scaling up deep reinforcement learning models has proven notoriously difficult due to network pathologies during training, motiv… (see more)ating various targeted interventions such as periodic reset and architectural advances such as layer normalization. Instead of pursuing more complex modifications, we show that introducing static network sparsity alone can unlock further scaling potential beyond their dense counterparts with state-of-the-art architectures. This is achieved through simple one-shot random pruning, where a predetermined percentage of network weights are randomly removed once before training. Our analysis reveals that, in contrast to naively scaling up dense DRL networks, such sparse networks achieve both higher parameter efficiency for network expressivity and stronger resistance to optimization challenges like plasticity loss and gradient interference. We further extend our evaluation to visual and streaming RL scenarios, demonstrating the consistent benefits of network sparsity.
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
PANDAS: Improving Many-shot Jailbreaking via Positive Affirmation, Negative Demonstration, and Adaptive Sampling
Avery Ma
Yangchen Pan
Amir-massoud Farahmand
Many-shot jailbreaking circumvents the safety alignment of large language models by exploiting their ability to process long input sequences… (see more). To achieve this, the malicious target prompt is prefixed with hundreds of fabricated conversational turns between the user and the model. These fabricated exchanges are randomly sampled from a pool of malicious questions and responses, making it appear as though the model has already complied with harmful instructions. In this paper, we present PANDAS: a hybrid technique that improves many-shot jailbreaking by modifying these fabricated dialogues with positive affirmations, negative demonstrations, and an optimized adaptive sampling method tailored to the target prompt's topic. Extensive experiments on AdvBench and HarmBench, using state-of-the-art LLMs, demonstrate that PANDAS significantly outperforms baseline methods in long-context scenarios. Through an attention analysis, we provide insights on how long-context vulnerabilities are exploited and show how PANDAS further improves upon many-shot jailbreaking.
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
PoisonBench: Assessing Language Model Vulnerability to Poisoned Preference Data
Tingchen Fu
Mrinank Sharma
Philip Torr
Shay B. Cohen
David M. Krueger
Fazl Barez
Preference learning is a central component for aligning current LLMs, but this process can be vulnerable to data poisoning attacks. To addre… (see more)ss this concern, we introduce PoisonBench, a benchmark for evaluating large language models' susceptibility to data poisoning during preference learning. Data poisoning attacks can manipulate large language model responses to include hidden malicious content or biases, potentially causing the model to generate harmful or unintended outputs while appearing to function normally. We deploy two distinct attack types across eight realistic scenarios, assessing 22 widely-used models. Our findings reveal concerning trends: (1) Scaling up parameter size does not always enhance resilience against poisoning attacks and the influence on model resilience varies among different model suites. (2) There exists a log-linear relationship between the effects of the attack and the data poison ratio; (3) The effect of data poisoning can generalize to extrapolated triggers that are not included in the poisoned data. These results expose weaknesses in current preference learning techniques, highlighting the urgent need for more robust defenses against malicious models and data manipulation.
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press
Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
Hanna Wallach
Meera Desai
A. Feder Cooper
Angelina Wang
Chad Atalla
Solon Barocas
Su Lin Blodgett
Alexandra Chouldechova
Emily Corvi
P. A. Dow
Jean Garcia-Gathright
A.R. Olteanu
Nicholas Pangakis
Stefanie Reed
Emily Sheng
Dan Vann
Jennifer Wortman Vaughan
Matthew Vogel
Hannah Washington
Abigail Z. Jacobs
The measurement tasks involved in evaluating generative AI (GenAI) systems are especially difficult, leading to what has been described as"a… (see more) tangle of sloppy tests [and] apples-to-oranges comparisons"(Roose, 2024). In this position paper, we argue that the ML community would benefit from learning from and drawing on the social sciences when developing and using measurement instruments for evaluating GenAI systems. Specifically, our position is that evaluating GenAI systems is a social science measurement challenge. We present a four-level framework, grounded in measurement theory from the social sciences, for measuring concepts related to the capabilities, behaviors, and impacts of GenAI. This framework has two important implications for designing and evaluating evaluations: First, it can broaden the expertise involved in evaluating GenAI systems by enabling stakeholders with different perspectives to participate in conceptual debates. Second, it brings rigor to both conceptual and operational debates by offering a set of lenses for interrogating the validity of measurement instruments and their resulting measurements.
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Position: Humanity Faces Existential Risk from Gradual Disempowerment
Jan Kulveit
Raymond Douglas
Nora Ammann
Deger Turan
David M. Krueger
David Duvenaud
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press
Position: Probabilistic Modelling is Sufficient for Causal Inference
Bruno Mlodozeniec
David M. Krueger
Richard E. Turner
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press
Position: The Right to AI
Rashid Mushkani
Hugo Berard
Allison Cohen
Shin Koseki
This position paper proposes a “Right to AI,” which asserts that individuals and communities should meaningfully participate in the deve… (see more)lopment and governance of the AI systems that shape their lives. Motivated by the increasing deployment of AI in critical domains and inspired by Henri Lefebvre’s concept of the “Right to the City,” we reconceptualize AI as a societal infrastructure, rather than merely a product of expert design. In this paper, we critically evaluate how generative agents, large-scale data extraction, and diverse cultural values bring new complexities to AI oversight. The paper proposes that grassroots participatory methodologies can mitigate biased outcomes and enhance social responsiveness. It asserts that data is socially produced and should be managed and owned collectively. Drawing on Sherry Arnstein’s Ladder of Citizen Participation and analyzing nine case studies, the paper develops a four-tier model for the Right to AI that situates the current paradigm and envisions an aspirational future. It proposes recommendations for inclusive data ownership, transparent design processes, and stakeholder-driven oversight. We also discuss market-led and state-centric alternatives and argue that participatory approaches offer a better balance between technical efficiency and democratic legitimacy.
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press
Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation
Mohit Pandey
Gopeshh Subbaraj
Artem Cherkasov
Martin Ester
Emmanuel Bengio
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Random Policy Evaluation Uncovers Policies of Generative Flow Networks
Haoran He
Emmanuel Bengio
Qingpeng Cai 0001
Ling Pan
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press
Rejecting Hallucinated State Targets during Planning
Mingde Zhao
Tristan Sylvain
Romain Laroche
Doina Precup
Yoshua Bengio
In planning processes of computational decision-making agents, generative or predictive models are often used as "generators" to propose "ta… (see more)rgets" representing sets of expected or desirable states. Unfortunately, learned models inevitably hallucinate infeasible targets that can cause delusional behaviors and safety concerns. We first investigate the kinds of infeasible targets that generators can hallucinate. Then, we devise a strategy to identify and reject infeasible targets by learning a target feasibility evaluator. To ensure that the evaluator is robust and non-delusional, we adopted a design choice combining off-policy compatible learning rule, distributional architecture, and data augmentation based on hindsight relabeling. Attaching to a planning agent, the designed evaluator learns by observing the agent's interactions with the environment and the targets produced by its generator, without the need to change the agent or its generator. Our controlled experiments show significant reductions in delusional behaviors and performance improvements for various kinds of existing agents.
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
doi.org
proceedings.mlr.press
Reward-Guided Prompt Evolving in Reinforcement Learning for LLMs
Ziyu Ye
Rishabh Agarwal
Tianqi Liu
Rishabh Joshi
Sarmishta Velury
Quoc V Le
Qijun Tan
Yuan Liu
2025-10-05
Proceedings of the 42nd International Conference on Machine Learning (published)
proceedings.mlr.press
proceedings.mlr.press