Publications

Large language models (LLMs) are increasingly applied to complex reasoning tasks that require executing several complex steps before receivi… (see more)ng any reward. Properly assigning credit to these steps is essential for enhancing model performance. Proximal Policy Optimization (PPO), a common reinforcement learning (RL) algorithm used for LLM finetuning, employs value networks to tackle credit assignment. However, recent approaches achieve strong results without it, raising questions about the efficacy of value networks in practice. In this work, we systematically evaluate the efficacy of value networks and reveal their significant shortcomings in reasoning-heavy LLM tasks, showing that they often produce poor estimate of expected return and barely outperform a random baseline when comparing alternative steps. This motivates our key question: Can improved credit assignment enhance RL training for LLMs? To address this, we propose VinePPO, a straightforward approach that leverages the flexibility of language environments to compute unbiased Monte Carlo-based estimates. Our method consistently outperforms PPO and other baselines across MATH and GSM8K datasets in less wall-clock time (up to 3.0x). Crucially, it achieves higher test accuracy for a given training accuracy, capturing more generalization signal per sample. These results emphasize the importance of accurate credit assignment in RL training of LLM.

2025-04-30

International Conference on Machine Learning (poster)

doi.org

proceedings.mlr.press

Caffeine induces age-dependent increases in brain complexity and criticality during sleep

Philipp Thölke

Maxine Arcand-Lavigne

Tarek Lajnef

Sonia Frenette

Julie Carrier

Karim Jerbi

Caffeine is the most widely consumed psychoactive stimulant worldwide. Yet important gaps persist in understanding its effects on the brain,… (see more) especially during sleep. We analyzed sleep electroencephalography (EEG) in 40 subjects, contrasting 200 mg of caffeine against a placebo condition, utilizing inferential statistics and machine learning. We found that caffeine ingestion led to an increase in brain complexity, a widespread flattening of the power spectrum’s 1/f-like slope, and a reduction in long-range temporal correlations. Being most prominent during non-rapid eye movement (NREM) sleep, these results suggest that caffeine shifts the brain towards a critical regime and more diverse neural dynamics. Interestingly, this was more pronounced in younger adults (20–27 years) compared to middle-aged participants (41–58 years) during rapid eye movement (REM) sleep, while no significant age effects were observed during NREM. Interpreting these data in the light of modeling and empirical work on EEG-derived measures of excitation-inhibition balance suggests that caffeine promotes a shift in brain dynamics towards increased neural excitation and closer proximity to a critical regime, particularly during NREM sleep.

2025-04-29

Communications Biology (published)

doi.org

JPerfEvo: A Tool for Tracking Method-Level Performance Changes in Java Projects

Kaveh Shahedi

Maxime Lamothe

Foutse Khomh

Heng Li

Performance regressions and improvements are common phenomena in software development, occurring periodically as software evolves and mature… (see more)s. When developers introduce new changes to a program’s codebase, unforeseen performance variations may arise. Identifying these changes at the method level, however, can be challenging due to the complexity and scale of modern codebases. In this work, we present JPerfEvo, a tool designed to automate the evaluation of the method-level performance impact of each code commit (i.e., the performance variations between the two versions before and after a commit). Leveraging the Java Microbenchmark Harness (JMH) module for benchmarking the modified methods, JPerfEvo instruments their execution and applies robust statistical evaluations to detect performance changes. The tool can classify these changes as performance improvements, regressions, or neutral (i.e., no change), with the change magnitude. We evaluated JPerfEvo on three popular and mature open-source Java projects, demonstrating its effectiveness in identifying performance changes throughout their development histories.

2025-04-27

IEEE Working Conference on Mining Software Repositories (published)

doi.org

Performance Smells in ML and Non-ML Python Projects: A Comparative Study

Franccois Belias

Leuson Da Silva

Foutse Khomh

Cyrine Zid

2025-04-27

ArXiv (preprint)

doi.org

arxiv.org

Proceedings of 1st Workshop on Advancing Artificial Intelligence through Theory of Mind

Mouad Abrini

Omri Abend

Dina M. Acklin

Henny Admoni

Gregor Aichinger

Nitay Alon

Zahra Ashktorab

Ashish Atreja

Moises Auron

Alexander Aufreiter

Raghav Awasthi

Soumya Banerjee

Joseph Barnby

Rhea Basappa

Severin Bergsmann

Djallel Bouneffouf

Patrick Callaghan

Marc Cavazza

Thierry Chaminade

Sonia Chernova … (see 88 more)

Mohamed Chetouan

Moumita Choudhury

Axel Cleeremans

J. Cywinski

Fabio Cuzzolin

Hokin Deng

N'yoma Diamond

C. D. Pasquasio

Guillaume Dumas

Max J. van Duijn

Mahapatra Dwarikanath

Qingying Gao

Ashok Goel

Rebecca R. Goldstein

Matthew C. Gombolay

Gabriel Enrique Gonzalez

Amar Halilovic

Tobias Halmdienst

Mahimul Islam

Julian Jara-Ettinger

Natalie Kastel

Renana Keydar

Ashish K. Khanna

Mahdi Khoramshahi

Jihyun Kim

Mihyeon Kim

Youngbin Kim

Senka Krivic

Nikita Krasnytskyi

Arun Kumar

Junehyoung Kwon

EunJu Lee

Shane Lee

Peter R. Lewis 0001

Xue Li

Yijiang Li

Michal Lewandowski

Nathan Lloyd

Matthew B. Luebbers

Dezhi Luo

Haiyun Lyu

Dwarikanath Mahapatra

Kamal Maheshwari

Mallika Mainali

P. Mathur

Patrick Mederitsch

Shuwa Miura

Manuel Preston de Miranda

Reuth Mirsky

Shreya Mishra

Nina M. Moorman

Katelyn Morrison

John Muchovej

Bernhard Nessler

Felix Nessler

Hieu Minh Jord Nguyen

Abby Ortego

F. Papay

Antoine Pasquali

Hamed Rahimi

C. Raghu

Amanda L. Royka

Stefan Sarkadi

Jaelle Scheuerman

Simon Schmid

Paul Schrater

Anik Sen

Zahra Sheikhbahaee

Ke Shi

Reid G. Simmons

Nishant Singh

Mason O. Smith

Ramira van der Meulen

Anthia Solaki

Haoran Sun

Viktor Szolga

Matthew E. Taylor

Travis Taylor

Sanne van Waveren

Juan David Vargas

R. Verbrugge

Eitan Wagner

Justin D. Weisz

Ximing Wen

William Yeoh

Wenlong Zhang

Michelle Zhao

Shlomo Zilberstein

2025-04-27

ArXiv (preprint)

doi.org

arxiv.org

Solving Combinatorial Pricing Problems using Embedded Dynamic Programming Models

Quang Minh Bui

Margarida Carvalho

José Neto

The combinatorial pricing problem (CPP) is a bilevel problem in which the leader maximizes their revenue by imposing tolls on certain items … (see more)that they can control. Based on the tolls set by the leader, the follower selects a subset of items corresponding to an optimal solution of a combinatorial optimization problem. To accomplish the leader's goal, the tolls need to be sufficiently low to discourage the follower from choosing the items offered by the competitors. In this paper, we derive a single-level reformulation for the CPP by rewriting the follower's problem as a longest path problem using a dynamic programming model, and then taking its dual and applying strong duality. We proceed to solve the reformulation in a dynamic fashion with a cutting plane method. We apply this methodology to 2 distinct dynamic programming models, namely, a novel formulation designated as selection diagram and the well-known decision diagram. We also produce numerical results to evaluate their performances across 3 different specializations of the CPP and a closely related problem that is the knapsack interdiction problem. Our results showcase the potential of the 2 proposed reformulations over the natural value function approach, expanding the set of tools to solve combinatorial bilevel programs.

2025-04-27

INFORMS Journal on Computing (published)

doi.org

arxiv.org

How Programmers Interact with Multimodal Software Documentation

Deeksha M. Arya

Jin L.C. Guo

Martin P. Robillard

There is a wide variety of online documentation to learn about a given software technology, and prior research has reported that programmers… (see more) must invest time and effort to identify one that best suits their need. We evaluated five modalities to present information that enable a software document to cater to the different presentation needs of programmers. We developed a prototype tutorial with these modalities on three topics in Java, namely, regular expressions, inheritance, and exception handling. We investigated how people interact with the modalities in the tutorial given a programming topic and a type of task. We conducted a survey study with 56 respondents and confirm that although text content is most useful for solving conceptual tasks, code examples support deeper comprehension of the underlying concepts. Furthermore, we report that respondents' contradicting preferences for the modalities suggest the need to have multiple alternatives in a software tutorial.

2025-04-26

2025 IEEE/ACM 18th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE) (published)

doi.org

A Learning-Based Framework for Fair and Scalable Solution Generation in Kidney Exchange Problems

William St-Arnaud

Margarida Carvalho

Golnoosh Farnadi

2025-04-26

TMLR (accepted)

openreview.net