Publications

Can Computational Reducibility Lead to Transferable Models for Graph Combinatorial Optimization?
Semih Cantürk
Thomas Sabourin
Frederik Wenkel
Michael Perlmutter
Guy Wolf
A key challenge in deriving unified neural solvers for combinatorial optimization (CO) is efficient generalization of models between a given… (voir plus) set of tasks to new tasks not used during the initial training process. To address it, we first establish a new model, which uses a GCON module as a form of expressive message passing together with energy-based unsupervised loss functions. This model achieves high performance (often comparable with state-of-the-art results) across multiple CO tasks when trained individually on each task. We then leverage knowledge from the computational reducibility literature to propose pretraining and fine-tuning strategies that transfer effectively (a) between MVC, MIS and MaxClique, and (b) in a multi-task learning setting that additionally incorporates MaxCut, MDS and graph coloring. Additionally, in a leave-one-out, multi-task learning setting, we observe that pretraining on all but one task almost always leads to faster convergence on the remaining task when fine-tuning while avoiding negative transfer. Our findings indicate that learning common representations across multiple graph CO problems is viable through the use of expressive message passing coupled with pretraining strategies that are informed by the polynomial reduction literature, thereby taking an important step towards enabling the development of foundational models for neural CO. We provide an open-source implementation of our work at https://github.com/semihcanturk/COPT-MT .
2026-04-29
International Conference on Machine Learning (accepté)
doi.org
openreview.net
Crisp: A Spectral-Based Interaction Strategy for Multivariate Time Series Forecasting
Binwu Wang
Gaoyun Lin
Jiaming Ma
Qihe Huang
Zhengyang Zhou
Xu Wang
Pengkun Wang
Yang Wang
Multivariate time series (MTS) forecasting critically depends on modeling inter-variable dependencies, yet existing paradigms face a trade-o… (voir plus)ff: channel-isolation strategies can suffer from information fragmentation in strongly coupled systems, whereas channel-interaction methods often introduce spurious interactions among irrelevant variables. To address this challenge, we propose Coherent Resonance Interaction with Spectral Priors (Crisp). Crisp adopts the principle that effective information exchange should occur only between variables with compatible oscillatory patterns. Concretely, we derive spectral priors in the frequency domain to construct dynamic resonance topologies. With a differentiable, adaptive, and strictly sparse blocking mechanism, Crisp forces attention weights for spectrally inconsistent neighbors to be exactly zero. In addition, we introduce a spectral-gated feature filtering module to refine variable representations using intrinsic spectral characteristics. Extensive experiments demonstrate that Crisp significantly outperforms 20+ baselines. Our code is available at Anonymous GitHub.
2026-04-29
International Conference on Machine Learning (accepté)
openreview.net
openreview.net
Distributionally Robust Set Representation Learning Under Inference-Time Element Corruption
Yankai Chen
Hanrong Zhang
Bowei He
Philip S. Yu
Xue (Steve) Liu
Standard Set Representation Learning methods typically excel on curated data but often overlook the challenge of Inference-time Element Corr… (voir plus)uption. This refers to scenarios where deployed models encounter element-level degradations, such as outliers or missing components, that may distort set representation and degrade performance. We propose SW-DRSO, a distributionally robust optimization framework tailored for sets. Rather than minimizing loss solely on observed training data, SW-DRSO optimizes a tractable surrogate of the worst-case expected loss over a family of plausible inference-time variations. We introduce a barycentric adversary that approximates the intractable search over corrupted sets by a differentiable training-time optimization over simplex weights. Extensive experiments across four tasks demonstrate that SW-DRSO effectively enhances robustness against corruption while maintaining high overall performance.
2026-04-29
International Conference on Machine Learning (accepté)
doi.org
openreview.net
Does a Hybrid Space-Aware Randomized Defense Improve Empirical and Certified Adversarial Robustness?
Joy Dhar
Manish Kumar Pandey
Behzad Bozorgtabar
Nayyar Zaidi
Wenyu Zhang
Wei-Hong Li
Tingting Mu
Dwarikanath Mahapatra
Mahsa Baktashmotlagh
Trung Le
Chen Chen
Sajib Mistry
Camila Gonzalez
Samira Ebrahimi Kahou
Lina Yao
Piotr Koniusz
Robert Burns Fisher
Dinh Phung
Bohyung Han
Nuno Vasconcelos … (voir 1 de plus)
Pietro Lio
We introduce Hybrid Space-aware Stochastic Convolution Attention Noise (HySCAN), a hybrid randomized defense that helps close the long-stand… (voir plus)ing gap between provable robustness under ℓ2 certificates and empirical robustness against strong ℓ∞ attacks, while maintaining strong generalization across diverse imaging benchmarks. HySCAN jointly explores complementary sources of stochasticity at both training and inference: (i) implicit weight-space randomness via stochastic-aware Random Weights, and (ii) explicit feature-space randomness via Stochastic Attention Noise Injection modules. By incorporating randomness at both the parameter and representation levels, HySCAN enables meaningful certified guarantees while improving empirical robustness in practice. Comprehensive experiments on diverse imaging datasets, e.g., CelebA, CIFAR-10, and CIFAR-100, ImageNet-1k, HAM10000, and NIH Chest X-ray, demonstrate that HySCAN outperforms existing certified and empirical defenses, improving certified robustness by up to ≈ 9.6% and empirical robustness by up to ≈ 5% without reducing clean accuracy.
2026-04-29
International Conference on Machine Learning (accepté)
openreview.net
openreview.net
EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings
Shiva Malay
Perampalli Shravan Nayak
Sagar Davasam
Srinivas Sunkara
Sai Rajeswar Mudumba
Jishnu Sethumadhavan Nair
Aman Tiwari
Sathwik Tejaswi Madhusudhan
Sridhar Krishna Nemala
The rapid evolution of Large Language Models (LLMs) has shifted their role from passive information providers to active agents capable of ex… (voir plus)ecuting complex workflows. However, the realization of a true "AI worker" is currently hindered by benchmarks that fail to capture the intricacy of professional environments, which demand long-horizon planning, complex tool usage, and adherence to strict access protocols. To bridge this gap, we introduce EnterpriseOps-Gym, a benchmark environment designed to evaluate agentic planning in realistic enterprise settings. EnterpriseOps-Gym provides: (i) 1,150 expert-curated tasks across eight interconnected domains (including HR, IT, Customer Service and productivity tools) that require managing persistent state and adhering to strict outcome-based verification logic; and (ii) a high-fidelity, containerized sandbox environment hosting 164 database tables and 512 functional tools. Our evaluation reveals critical limitations in state-of-the-art models: even the top-performing Claude Sonnet~4.5 achieves only 34.1\% success, struggling significantly with planning consistency, error recovery, and policy constraints. Furthermore, we observe that agents frequently fail to refuse infeasible tasks, leading to unintended and potentially harmful side effects on the system. These findings indicate that current agents are not yet ready for enterprise deployment. By releasing EnterpriseOps-Gym, we provide a concrete testbed to advance the reliability of autonomous agents in professional workflows.
2026-04-29
International Conference on Machine Learning (accepté)
doi.org
openreview.net
FedFit: Federated Dynamic Sparse Training via Fisher Information scoring
Meng Bi
Hong Huang
Jinlong Song
Charles Wang
Chengming Hu
Xi Chen
Ting Yu
Xue Liu
Cross-device Federated Learning (FL) is frequently bottlenecked by the prohibitive memory and communication costs of training deep neural ne… (voir plus)tworks on resource-constrained edge hardware. While federated dynamic sparse training aims to alleviate these costs by adjusting sparse structures during training, existing methods rely on magnitude-based heuristics that are fundamentally ill-suited for the non-convergent, heterogeneous environments inherent to FL. To address this challenge, we propose FedFit, a federated dynamic sparse training framework that replaces simple heuristics with optimization-centric criteria for structure adjustment. By leveraging a second-order approximation of the loss landscape via the Fisher Information Matrix, FedFit enables precise and efficient structure adjustment without the overhead of explicit Hessian computation. Empirical evaluations across computer vision and natural language processing benchmarks demonstrate that FedFit significantly narrows the sparse-to-dense accuracy gap, outperforming state-of-the-art methods while maintaining high communication efficiency. Our code is available at https://github.com/Serena-28/Fedfit.git.
2026-04-29
International Conference on Machine Learning (accepté)
openreview.net
openreview.net
From Lyapunov Analysis to Algorithm Design in two-sided PL Minimax Optimization
Mansi Rankawat
Michael Muehlebach
Simon Lacoste-Julien
Damien Scieur
We derive algorithms for smooth nonconvex nonconcave minimax optimization and establish linear convergence rates for problems that satisfy t… (voir plus)he two-sided Polyak-Lojasiewicz (PL) inequality. At the core of our approach is the observation that Lyapunov functions can be used not only to certify convergence a posteriori, but also to design algorithms. By replacing an idealized, intractable Lyapunov function with a computable surrogate based on gradient information, we derive TALDA (Tri-Action Lyapunov Descent Ascent), a single-loop algorithm that enforces Lyapunov descent by construction. TALDA guarantees linear convergence under the two-sided PL condition, with a rate that depends explicitly on the cross-smoothness constant. This recovers existing worst-case guarantees while yielding sharper convergence rates in weakly coupled min–max problems.
2026-04-29
International Conference on Machine Learning (accepté)
openreview.net
openreview.net
Large Language Models Are Good Term Extractors: A Systematic Evaluation
Ayla Rigouts Terryn
2026-04-29
Language Resources and Evaluation Conference (publié)
doi.org
LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs
Benno Krojer
Shravan Nayak
Oscar Mañas
Vaibhav Adlakha
Desmond Elliott
Siva Reddy
Marius Mosbach
Transforming a large language model (LLM) into a Vision-Language Model (VLM) can be achieved by mapping the visual tokens from a vision enco… (voir plus)der into the embedding space of an LLM. Intriguingly, this mapping can be as simple as a shallow MLP transformation. To understand why LLMs can so readily process visual tokens, we need interpretability methods that reveal what is encoded in the visual token representations at every layer of LLM processing. In this work, we introduce LatentLens, a novel approach for mapping latent representations to descriptions in natural language. LatentLens works by encoding a large text corpus and storing contextualized token representations for each token in that corpus. Visual token representations are then compared to their contextualized textual representations, with the top-k nearest neighbor representations providing descriptions of the visual token. We evaluate this method on 10 different VLMs, showing that commonly used methods, such as LogitLens, substantially underestimate the interpretability of visual tokens. With LatentLens instead, the majority of visual tokens are interpretable across all studied models and all layers. Qualitatively, we show that the descriptions produced by LatentLens are semantically meaningful and provide more fine-grained interpretations for humans compared to individual tokens. More broadly, our findings contribute new evidence on the alignment between vision and language representations, opening up new directions for analyzing latent representations.
2026-04-29
International Conference on Machine Learning (accepté)
doi.org
openreview.net
MIRA: A Score for Conditional Distribution Accuracy and Model Comparison
Sammy Sharief
Justine Zeghal
Gabriel Missael Barco
Pablo Lemos
Yashar Hezaveh
Laurence Perreault-Levasseur
We present Mira, a method for estimating the expected probability that samples from a candidate conditional distribution match the true, unk… (voir plus)nown conditional distribution, for which only data-label pairs are available. We derive theoretical bounds obtained when the candidate distribution matches the true one and when the conditional distributions are independent. This framework thus enables model comparison by quantifying the alignment between the conditional distribution of a candidate model and the data-label pairs of the true model. Consequently, Mira enables Bayesian model comparison through direct posterior validation, bypassing the challenging evidence computation. We demonstrate its effectiveness across several toy problems and Bayesian inference tasks.
2026-04-29
International Conference on Machine Learning (spotlight)
doi.org
openreview.net
Operationalising the Superficial Alignment Hypothesis via Task Complexity
Tomás Vergara-Browne
Darshan Patil
Ivan Titov
Siva Reddy
Tiago Pimentel
Marius Mosbach
The superficial alignment hypothesis (SAH) posits that large language models learn most of their knowledge during pre-training, and that pos… (voir plus)t-training merely surfaces this knowledge. The SAH, however, lacks a precise definition, which has led to (i) different and seemingly orthogonal arguments supporting it, and (ii) important critiques to it. We propose a new metric called task complexity: the length of the shortest program that achieves a target performance on a task. In this framework, the SAH simply claims that pre-trained models drastically reduce the complexity of achieving high performance on many tasks. Our definition unifies prior arguments supporting the SAH, interpreting them as different strategies to find such short programs. Experimentally, we estimate the task complexity of mathematical reasoning, machine translation, and instruction following; we then show that these complexities can be remarkably low when conditioned on a pre-trained model. Further, we find that pre-training enables access to strong performances on our tasks, but it can require programs of gigabytes of length to access them. Post-training, on the other hand, collapses the complexity of reaching this same performance by several orders of magnitude. Overall, our results highlight that task adaptation often requires surprisingly little information -- often just a few kilobytes.
2026-04-29
International Conference on Machine Learning (accepté)
doi.org
openreview.net
Overcoming the Modality Gap in Context-Aided Forecasting
Vincent Zhihao Zheng
Étienne Marcotte
Arjun Ashok
Andrew Robert Williams
Lijun Sun
Alexandre Drouin
Valentina Zantedeschi
Context-aided forecasting (CAF) holds promise for integrating domain knowledge and forward-looking information, enabling AI systems to surpa… (voir plus)ss traditional statistical methods. However, recent empirical studies reveal a puzzling gap: multimodal models often fail to outperform their unimodal counterparts. We hypothesize that this underperformance stems from poor context quality in existing datasets, as verification is challenging. To address these limitations, we introduce a semi-synthetic data augmentation method that generates contexts both descriptive of temporal dynamics and verifiably complementary to numerical histories. This approach enables massive-scale dataset creation, resulting in CAF-7M, a corpus of 7 million context-augmented time series windows, including a rigorously verified test set. We demonstrate that semi-synthetic pre-training transfers effectively to real-world evaluation, and show clear evidence of context utilization. Our results suggest that dataset quality, rather than architectural limitations, has been the primary bottleneck in context-aided forecasting.
2026-04-29
International Conference on Machine Learning (accepté)
doi.org
openreview.net