Publications

From Memorization to Parameter Interference: How Overtraining Experts Harms Model Merging
Stefan Horoi
Guy Wolf
Eugene Belilovsky
Gintare Karolina Dziugaite
Modern deep learning is increasingly characterized by the use of open-weight foundation models that can be fine-tuned on specialized dataset… (voir plus)s. This has led to a proliferation of expert models and adapters, often shared via platforms like HuggingFace and AdapterHub. Model merging has recently emerged as an effective way to leverage these existing resources, enabling the composition of capabilities from different model checkpoints. A natural pipeline has thus formed to harness the benefits of transfer learning and amortize sunk training costs: models are pre-trained on general data, fine-tuned on specific tasks, and then multiple checkpoints are merged to obtain a more capable model. A prevailing assumption is that improvements at one stage of this pipeline propagate downstream, leading to gains at subsequent steps. In this work, we challenge that assumption by examining how expert fine-tuning affects model merging. We show that long fine-tuning of experts that optimizes for their individual performance leads to degraded merging performance across vision and language modalities, multiple model scales, and both fully fine-tuned and LoRA-adapted models. We trace this degradation to the memorization of a small set of difficult examples that dominate late fine-tuning steps. This causes negative parameter interference and encodes knowledge that is forgotten during merging. Finally, we demonstrate that task-dependent aggressive early stopping strategies can significantly improve model merging performance.
2025-12-31
International Conference on Machine Learning (Accept (regular))
openreview.net
openreview.net
From Where Things Are to What They Are For: Benchmarking Spatial–Functional Intelligence in Multimodal LLMs
Le Zhang
Jihan Yang
Soundarya Krishnan
Jimit Majmudar
Xiou Ge
Prasoon Puri
Prathamesh Saraf
Shruti Bhargava
Dhivya Piraviperumal
Yinan Ling
Cindy Pan
Hong Yu
Aishwarya Agrawal
Bo-Hsiang Tseng
Human level agentic intelligence transcends low-level geometric perception, evolving from knowing where things are to understanding what the… (voir plus)y are for. While existing benchmarks effectively evaluate this foundational geometric perception capabilites of multimodal LLMs, they fall short of probing the higher-order cognitive abilities essential for grounded intelligence. To bridge this gap, we introduce the Spatial-Functional Intelligence Benchmark (SFI-Bench), a video-based benchmark with over 1500 expert-annotated questions derived from diverse, egocentric indoor video scans. SFI-Bench is designed to systematically evaluate two complementary dimensions of advanced reasoning: 1) Structured Spatial Reasoning, understanding complex layouts and forming coherent spatial representations, and 2) Functional Reasoning, inferring object affordances and context-dependent utility. Its tasks, including conditional counting, multi-hop relational reasoning, functional pairing, and knowledge-grounded troubleshooting, directly challenge a model's ability to integrate perception, memory, and inference. Our experiments reveal that current MLLMs consistently struggle to integrate spatial memory with functional and external knowledge, highlighting a critical bottleneck. SFI-Bench thus provides an essential tool for measuring and driving progress towards more cognitively capable and truly grounded multimodal agents.
Gait training combined with transcutaneous spinal stimulation to enhance lower limbs motor recovery in people with spinal cord injury: Pilot Study
Nicolas Hoang Quang
Marianne Cossette-Levasseur
Sammy-Jo Beauregard-Veillette
Nancy Dubé
El-Mehdi Meftah
Héloïse Bourgeois
Nok-Yeung Law
Amedeo Ceglia
Marco Bonizzato
Marina Martinez
Diana Zidarov
Dorothy Barthélemy
2025-12-31
Neuromodulation: Technology at the Neural Interface (publié)
doi.org
Generative Adversarial Post-Training Mitigates Reward Hacking in Live Human-AI Music Interaction
Yusong Wu
Stephen Brade
Aleksandra Teng Ma
Tia-Jane Fowler
Enning Yang
Berker Banar
Aaron Courville
Natasha Jaques
Cheng-Zhi Anna Huang
Most applications of generative AI involve a sequential interaction in which a person inputs a prompt and waits for a response, and where re… (voir plus)action time and adaptivity are not important factors. In contrast, live jamming is a collaborative interaction that requires real-time coordination and adaptation without access to the other player’s future moves, while preserving diversity to sustain a creative flow. Reinforcement learning post-training enables effective adaptation through on-policy interaction, yet it often reduces output diversity by exploiting coherence-based rewards. This collapse, known as ``reward hacking'', affects many RL post-training pipelines, but is especially harmful in live jamming, where musical creativity relies on dynamic variation and mutual responsiveness. In this paper, we propose a novel adversarial training method on policy-generated trajectories to mitigate reward hacking in RL post-training for melody-to-chord accompaniment. A co-evolving discriminator separates policy trajectories from the data distribution, while the policy maximizes the discriminator output in addition to coherence rewards to prevent collapse to trivial outputs. We evaluate accompaniment quality and output diversity in simulation with both fixed test melodies and learned melody agents, and we conduct a user study with the model deployed in a real-time interactive system with expert musicians. Quantitative evaluation and user feedback demonstrate improved output diversity, harmonic coherence, adaptation speed and user agency. Our results demonstrate a simple yet effective method to mitigate reward hacking in RL post-training of generative sequence models.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
Generative Floor Plan Design with LLMs via Reinforcement Learning with Verifiable Rewards
Luis Lara
Aristides Milios
Zhi Hao Luo
Aditya Sharma
Ge Ya Luo
Christopher Beckham
Florian Golemo
Christopher Pal
An AI system for professional floor plan design needs to be able to precisely control room dimensions and areas (quantitative constraints), … (voir plus)while also balancing functional considerations and design aesthetics. Existing generative approaches focus primarily on respecting the requested connectivity between rooms, but do not support generating floor plans with numerical constraints. We introduce a text‑based floor plan generation approach that fine-tunes a large language model (LLM) on real plans and then applies reinforcement learning with verifiable rewards (RLVR) to enforce both numerical (areas, dimensions) and spatial (topological) constraints. Furthermore, we design a set of constraint adherence metrics to measure how generated floor plans align with user-defined constraints systematically. Our model generates floor plans that satisfy numerical constraints and outperforms existing methods on realism, compatibility, and diversity scores. Specifically, our approach leads to an up to 94\% reduction in compatibility score. Our results demonstrate that LLMs can effectively handle quantitative constraints in structured design tasks, suggesting broader applications for text-based generative modeling.
2025-12-31
Findings of the Association for Computational Linguistics (publié)
doi.org
openreview.net
Gistify: Codebase-Level Understanding via Runtime Execution
Hyunji Lee
Minseon Kim
Chinmay Singh
Matheus Pereira
Atharv Sonwane
Isadora White
Elias Stengel-Eskin
Mohit Bansal
Zhengxiang Shi
Alessandro Sordoni
Marc-Alexandre Côté
Eric Yuan
Lucas Caccia
As coding agents are increasingly deployed in large codebases, the need to automatically design challenging, codebase-level evaluation is ce… (voir plus)ntral. We propose Gistify, a task where a coding LLM must create a single, minimal, self-contained file that can reproduce a specific functionality of a codebase. The coding LLM is given full access to a codebase along with a specific entrypoint (e.g., a python command), and the generated file must replicate the output of the same command ran under the full codebase, while containing only the essential components necessary to execute the provided command. Success on Gistify requires both structural understanding of the codebase, accurate modeling of its execution flow as well as the ability to produce potentially large code patches. Our findings show that current state-of-the-art models struggle to reliably solve Gistify tasks, especially ones with long executions traces.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
GitChameleon 2.0: Evaluating AI Code Generation Against Python Library Version Incompatibilities
Diganta Misra
Nizar Islah
Victor May
Brice Rauby
Zihan Wang
Justine Gehring
Antonio Orvieto
Muawiz Chaudhary
Eilif B. Muller
Irina Rish
Samira Ebrahimi Kahou
Massimo Caccia
The rapid evolution of software libraries poses a considerable hurdle for code generation, necessitating continuous adaptation to frequent v… (voir plus)ersion updates while preserving backward compatibility. While existing code evolution benchmarks provide valuable insights, they typically lack execution-based evaluation for generating code compliant with specific library versions. To address this, we introduce GitChameleon 2.0, a novel, meticulously curated dataset comprising 328 Python code completion problems, each conditioned on specific library versions and accompanied by executable unit tests. GitChameleon 2.0 rigorously evaluates the capacity of contemporary large language models (LLMs), LLM-powered agents, code assistants, and RAG systems to perform version-conditioned code generation that demonstrates functional accuracy through execution. Our extensive evaluations indicate that state-of-the-art systems encounter significant challenges with this task; enterprise models achieving baseline success rates in the 48-51% range, underscoring the intricacy of the problem. By offering an execution-based benchmark emphasizing the dynamic nature of code libraries, GitChameleon 2.0 enables a clearer understanding of this challenge and helps guide the development of more adaptable and dependable AI code generation methods. We make the dataset and evaluation code publicly available at https://github.com/mrcabbage972/GitChameleonBenchmark.
2025-12-31
Association for Computational Linguistics (publié)
doi.org
openreview.net
GraphOmni: A Comprehensive and Extensible Benchmark Framework for Large Language Models on Graph-theoretic Tasks
Hao Xu
Xiangru Jian
Xinjian Zhao
Wei Pang
Chao Zhang
Suyuchen Wang
Qixin Zhang
Zhengyuan Dong
Joao Monteiro
Bang Liu
Qiuzhuang Sun
Tianshu Yu
This paper introduces GraphOmni, a comprehensive benchmark designed to evaluate the reasoning capabilities of LLMs on graph-theoretic tasks … (voir plus)articulated in natural language. GraphOmni spans diverse graph types, serialization formats, and prompting schemes, substantially extending upon prior efforts in both scope and depth. Through systematic evaluation, we uncover critical interactions among these dimensions, revealing their decisive impact on model performance. Our experiments show that state-of-the-art closed-source models such as Claude-3.5 and o4-mini consistently lead overall, yet still leave considerable headroom, while open-source models display pronounced sensitivity to various design choices. Beyond the standard scope, larger graphs, real-world graphs, and additional NP-hard tasks are further discussed. We further analyze efficiency via output token usage, highlighting cost–accuracy trade-offs, and introduce a reinforcement learning-based optimizer that adaptively selects factor combinations, reducing evaluation cost by 75\% while retaining strong accuracy. This flexible and extensible benchmark not only deepens understanding of LLM performance on structured graph reasoning but also establishes a robust foundation for advancing model design and evaluation. The code and datasets are available at https://anonymous.4open.science/r/ID-14092.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
Grounding Computer Use Agents on Human Demonstrations
Aarash Feizi
Shravan Nayak
Xiangru Jian
Kevin Qinghong Lin
Kaixin Li
Rabiul Awal
Xing Han Lu
Johan Obando-Ceron
Juan A. Rodriguez
Nicolas Chapados
David Vázquez
Adriana Romero-Soriano
Reihaneh Rabbany
Perouz Taslakian
Christopher Pal
Spandana Gella
Sai Rajeswar
Building reliable computer-use agents requires grounding: accurately connecting natural language instructions to the correct on-screen eleme… (voir plus)nts. While large datasets exist for web and mobile interactions, high-quality resources for desktop environments are limited. To address this gap, we introduce GroundCUA, a large-scale desktop grounding dataset built from expert human demonstrations. It covers 87 applications across 12 categories and includes 56K screenshots, with every on-screen element carefully annotated for a total of over 3.56M human-verified annotations. From these demonstrations, we generate diverse instructions that capture a wide range of real-world tasks, providing high-quality data for model training. Using GroundCUA, we develop the GroundNext family of models that map instructions to their target UI elements. At both 3B and 7B scales, GroundNext achieves state-of-the-art results across five benchmarks using supervised fine-tuning, while requiring less than one-tenth the training data of prior work. Reinforcement learning post-training further improves performance. These results demonstrate the critical role of high-quality, expert-driven datasets in advancing general-purpose computer-use agents.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
Hierarchical Retrieval at Scale: Bridging Transparency and Efficiency
Shubham Gupta
Zichao Li
Tianyi Chen
Cem Subakan
Siva Reddy
Perouz Taslakian
Valentina Zantedeschi
Information retrieval is a core component of many intelligent systems as it enables conditioning of outputs on new and large-scale datasets.… (voir plus) While effective, the standard practice of encoding data into high-dimensional representations for similarity search entails large memory and compute footprints, and also makes it hard to inspect the inner workings of the system. Hierarchical retrieval methods offer an interpretable alternative by organizing data at multiple granular levels, yet do not match the efficiency and performance of flat retrieval approaches. In this paper, we propose ReTreever, a tree-based method that makes hierarchical retrieval viable at scale by directly optimizing its structure for retrieval performance while naturally providing transparency through meaningful semantic groupings. Our method offers the flexibility to balance cost and utility by indexing data using representations from any tree level. We show that ReTreever delivers strong coarse (intermediate levels) and fine representations (terminal level), while achieving the highest retrieval accuracy at the lowest latency among hierarchical methods. These results demonstrate that this family of techniques is viable in practical applications.
2025-12-31
International Conference on Machine Learning (Accept (regular))
doi.org
openreview.net
Hierarchical Value-Decomposed Offline Reinforcement Learning for Whole-Body Control
Zhilong Zhang
Yunpeng Mei
Xinghao Du
Hongjie Cao
Haonan Wang
Pengyuan Min
Chenyu Wang
Pengfei Chen
Chenbo Xin
Yijie Wang
Wenyu Luo
Yihao Sun
Yidi Wang
Lei Yuan
Gang Wang
Yang Yu
Scaling imitation learning to high-DoF whole-body robots is fundamentally limited by the \textbf{curse of dimensionality} and the prohibitiv… (voir plus)e cost of collecting expert demonstrations. We argue that the core bottleneck is paradigmatic: real-world supervision for whole-body control is inherently imperfect, yet most methods assume expert data. To overcome this, we propose \textbf{HVD} (Hierarchical Value-Decomposed Offline Reinforcement Learning), an offline reinforcement learning framework that learns effective policies directly from suboptimal, reward-labeled trajectories. HVD structures the value function along the robot’s kinematic hierarchy and over temporal chunks, enabling precise credit assignment in long-horizon, high-dimensional tasks. Built on a Transformer-based architecture, HVD supports \textit{multi-modal} and \textit{multi-task} learning, allowing flexible integration of diverse sensory inputs for policy learning. To enable realistic evaluation and training, we further introduce \textbf{WB-50}, a 50-hour dataset of teleoperated and policy rollout trajectories annotated with rewards and preserving natural imperfections — including partial successes, corrections, and failures. Experiments show HVD significantly outperforms existing baselines in success rate across complex whole-body tasks. Our results suggest that effective policy learning for high-DoF systems can emerge not from perfect demonstrations, but from structured learning over realistic, imperfect data.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
h-MINT: Modeling Pocket-Ligand Binding with Hierarchical Molecular Interaction Network
Yanru Qu
Yijie Zhang
Wenjuan Tan
Xiangzhe Kong
Xiangxin Zhou
Chaoran Cheng
Mathieu Blanchette
Jiaxuan You
Ge Liu
Accurate molecular representations are critical for drug discovery, and a central challenge lies in capturing the chemical environment of mo… (voir plus)lecular fragments, as key interactions, such as H-bond and π stacking—occur only under specific local conditions. Most existing approaches represent molecules as atom-level graphs; however, individual atoms cannot express stereochemistry, lone pairs, conjugation, and other complex features. Fragment-based methods (e.g., principal subgraph or functional group libraries) fail to preserve essential information such as chirality, aromatic bond integrity, and ionic states. This work addresses these limitations from two aspects. (i) **OverlapBPE tokenization**. We propose a novel data-driven molecule tokenization method. Unlike existing approaches, our method allows overlapping fragments, reflecting the inherently fuzzy boundaries of small-molecule substructures and, together with enriched chemical information at the token level, thereby preserving a more complete chemical context. (ii) **h- MINT model**. We develop a hierarchical molecular interaction network capable of jointly modeling drug–target interactions at both atom and fragment levels. By supporting fragment overlaps, the model naturally accommodates the many-to- many atom–fragment mappings introduced by the OverlapBPE scheme. Extensive evaluation against state-of-the-art methods shows our method improves binding affinity prediction by 2-4% Pearson/Spearman correlation on PDBBind and LBA, enhances virtual screening by 1-3% in key metrics on DUD-E and LIT-PCBA, and achieves the best overall HTS performance on PubChem assays. Further analysis demonstrates that our method effectively captures interactive information while maintaining good generalization.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net