Publications

Can Computational Reducibility Lead to Transferable Models for Graph Combinatorial Optimization?

Michael Perlmutter

A key challenge in deriving unified neural solvers for combinatorial optimization (CO) is efficient generalization of models between a given… (see more) set of tasks to new tasks not used during the initial training process. To address it, we first establish a new model, which uses a GCON module as a form of expressive message passing together with energy-based unsupervised loss functions. This model achieves high performance (often comparable with state-of-the-art results) across multiple CO tasks when trained individually on each task. We then leverage knowledge from the computational reducibility literature to propose pretraining and fine-tuning strategies that transfer effectively (a) between MVC, MIS and MaxClique, and (b) in a multi-task learning setting that additionally incorporates MaxCut, MDS and graph coloring. Additionally, in a leave-one-out, multi-task learning setting, we observe that pretraining on all but one task almost always leads to faster convergence on the remaining task when fine-tuning while avoiding negative transfer. Our findings indicate that learning common representations across multiple graph CO problems is viable through the use of expressive message passing coupled with pretraining strategies that are informed by the polynomial reduction literature, thereby taking an important step towards enabling the development of foundational models for neural CO. We provide an open-source implementation of our work at https://github.com/semihcanturk/COPT-MT .

2026-04-29

International Conference on Machine Learning (accepted)

doi.org

openreview.net

Crisp: A Spectral-Based Interaction Strategy for Multivariate Time Series Forecasting

Binwu Wang

Gaoyun Lin

Jiaming Ma

Qihe Huang

Zhengyang Zhou

Xu Wang

Pengkun Wang

Yang Wang

Multivariate time series (MTS) forecasting critically depends on modeling inter-variable dependencies, yet existing paradigms face a trade-o… (see more)ff: channel-isolation strategies can suffer from information fragmentation in strongly coupled systems, whereas channel-interaction methods often introduce spurious interactions among irrelevant variables. To address this challenge, we propose Coherent Resonance Interaction with Spectral Priors (Crisp). Crisp adopts the principle that effective information exchange should occur only between variables with compatible oscillatory patterns. Concretely, we derive spectral priors in the frequency domain to construct dynamic resonance topologies. With a differentiable, adaptive, and strictly sparse blocking mechanism, Crisp forces attention weights for spectrally inconsistent neighbors to be exactly zero. In addition, we introduce a spectral-gated feature filtering module to refine variable representations using intrinsic spectral characteristics. Extensive experiments demonstrate that Crisp significantly outperforms 20+ baselines. Our code is available at Anonymous GitHub.

2026-04-29

International Conference on Machine Learning (accepted)

openreview.net

Distributionally Robust Set Representation Learning Under Inference-Time Element Corruption

Yankai Chen

Hanrong Zhang

Bowei He

Philip S. Yu

Xue (Steve) Liu

Standard Set Representation Learning methods typically excel on curated data but often overlook the challenge of Inference-time Element Corr… (see more)uption. This refers to scenarios where deployed models encounter element-level degradations, such as outliers or missing components, that may distort set representation and degrade performance. We propose SW-DRSO, a distributionally robust optimization framework tailored for sets. Rather than minimizing loss solely on observed training data, SW-DRSO optimizes a tractable surrogate of the worst-case expected loss over a family of plausible inference-time variations. We introduce a barycentric adversary that approximates the intractable search over corrupted sets by a differentiable training-time optimization over simplex weights. Extensive experiments across four tasks demonstrate that SW-DRSO effectively enhances robustness against corruption while maintaining high overall performance.

2026-04-29

International Conference on Machine Learning (accepted)

doi.org

openreview.net

Does a Hybrid Space-Aware Randomized Defense Improve Empirical and Certified Adversarial Robustness?

Joy Dhar

Manish Kumar Pandey

Behzad Bozorgtabar

Nayyar Zaidi

Wenyu Zhang

Wei-Hong Li

Tingting Mu

Dwarikanath Mahapatra

Mahsa Baktashmotlagh

Trung Le

Chen Chen

Sajib Mistry

Camila Gonzalez

Samira Ebrahimi Kahou

Lina Yao

Piotr Koniusz

Robert Burns Fisher

Dinh Phung

Bohyung Han

Nuno Vasconcelos … (see 1 more)

Pietro Lio

We introduce Hybrid Space-aware Stochastic Convolution Attention Noise (HySCAN), a hybrid randomized defense that helps close the long-stand… (see more)ing gap between provable robustness under ℓ2 certificates and empirical robustness against strong ℓ∞ attacks, while maintaining strong generalization across diverse imaging benchmarks. HySCAN jointly explores complementary sources of stochasticity at both training and inference: (i) implicit weight-space randomness via stochastic-aware Random Weights, and (ii) explicit feature-space randomness via Stochastic Attention Noise Injection modules. By incorporating randomness at both the parameter and representation levels, HySCAN enables meaningful certified guarantees while improving empirical robustness in practice. Comprehensive experiments on diverse imaging datasets, e.g., CelebA, CIFAR-10, and CIFAR-100, ImageNet-1k, HAM10000, and NIH Chest X-ray, demonstrate that HySCAN outperforms existing certified and empirical defenses, improving certified robustness by up to ≈ 9.6% and empirical robustness by up to ≈ 5% without reducing clean accuracy.

2026-04-29

International Conference on Machine Learning (accepted)

openreview.net

EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings

Shiva Malay

Perampalli Shravan Nayak

Sagar Davasam

Srinivas Sunkara

Sai Rajeswar Mudumba

Jishnu Sethumadhavan Nair

Aman Tiwari

Sathwik Tejaswi Madhusudhan

Sridhar Krishna Nemala

The rapid evolution of Large Language Models (LLMs) has shifted their role from passive information providers to active agents capable of ex… (see more)ecuting complex workflows. However, the realization of a true "AI worker" is currently hindered by benchmarks that fail to capture the intricacy of professional environments, which demand long-horizon planning, complex tool usage, and adherence to strict access protocols. To bridge this gap, we introduce EnterpriseOps-Gym, a benchmark environment designed to evaluate agentic planning in realistic enterprise settings. EnterpriseOps-Gym provides: (i) 1,150 expert-curated tasks across eight interconnected domains (including HR, IT, Customer Service and productivity tools) that require managing persistent state and adhering to strict outcome-based verification logic; and (ii) a high-fidelity, containerized sandbox environment hosting 164 database tables and 512 functional tools. Our evaluation reveals critical limitations in state-of-the-art models: even the top-performing Claude Sonnet~4.5 achieves only 34.1\% success, struggling significantly with planning consistency, error recovery, and policy constraints. Furthermore, we observe that agents frequently fail to refuse infeasible tasks, leading to unintended and potentially harmful side effects on the system. These findings indicate that current agents are not yet ready for enterprise deployment. By releasing EnterpriseOps-Gym, we provide a concrete testbed to advance the reliability of autonomous agents in professional workflows.

2026-04-29

International Conference on Machine Learning (accepted)

doi.org

openreview.net

FedFit: Federated Dynamic Sparse Training via Fisher Information scoring

Meng Bi

Hong Huang

Jinlong Song

Charles Wang

Chengming Hu

Xi Chen

Ting Yu

Xue Liu

Cross-device Federated Learning (FL) is frequently bottlenecked by the prohibitive memory and communication costs of training deep neural ne… (see more)tworks on resource-constrained edge hardware. While federated dynamic sparse training aims to alleviate these costs by adjusting sparse structures during training, existing methods rely on magnitude-based heuristics that are fundamentally ill-suited for the non-convergent, heterogeneous environments inherent to FL. To address this challenge, we propose FedFit, a federated dynamic sparse training framework that replaces simple heuristics with optimization-centric criteria for structure adjustment. By leveraging a second-order approximation of the loss landscape via the Fisher Information Matrix, FedFit enables precise and efficient structure adjustment without the overhead of explicit Hessian computation. Empirical evaluations across computer vision and natural language processing benchmarks demonstrate that FedFit significantly narrows the sparse-to-dense accuracy gap, outperforming state-of-the-art methods while maintaining high communication efficiency. Our code is available at https://github.com/Serena-28/Fedfit.git.

2026-04-29

International Conference on Machine Learning (accepted)

openreview.net

From Lyapunov Analysis to Algorithm Design in two-sided PL Minimax Optimization

Mansi Rankawat

Michael Muehlebach

Simon Lacoste-Julien

Damien Scieur

We derive algorithms for smooth nonconvex nonconcave minimax optimization and establish linear convergence rates for problems that satisfy t… (see more)he two-sided Polyak-Lojasiewicz (PL) inequality. At the core of our approach is the observation that Lyapunov functions can be used not only to certify convergence a posteriori, but also to design algorithms. By replacing an idealized, intractable Lyapunov function with a computable surrogate based on gradient information, we derive TALDA (Tri-Action Lyapunov Descent Ascent), a single-loop algorithm that enforces Lyapunov descent by construction. TALDA guarantees linear convergence under the two-sided PL condition, with a rate that depends explicitly on the cross-smoothness constant. This recovers existing worst-case guarantees while yielding sharper convergence rates in weakly coupled min–max problems.

2026-04-29

International Conference on Machine Learning (accepted)

openreview.net

Large Language Models Are Good Term Extractors: A Systematic Evaluation

Ayla Rigouts Terryn

2026-04-29

Language Resources and Evaluation Conference (published)

doi.org

LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs

Transforming a large language model (LLM) into a Vision-Language Model (VLM) can be achieved by mapping the visual tokens from a vision enco… (see more)der into the embedding space of an LLM. Intriguingly, this mapping can be as simple as a shallow MLP transformation. To understand why LLMs can so readily process visual tokens, we need interpretability methods that reveal what is encoded in the visual token representations at every layer of LLM processing. In this work, we introduce LatentLens, a novel approach for mapping latent representations to descriptions in natural language. LatentLens works by encoding a large text corpus and storing contextualized token representations for each token in that corpus. Visual token representations are then compared to their contextualized textual representations, with the top-k nearest neighbor representations providing descriptions of the visual token. We evaluate this method on 10 different VLMs, showing that commonly used methods, such as LogitLens, substantially underestimate the interpretability of visual tokens. With LatentLens instead, the majority of visual tokens are interpretable across all studied models and all layers. Qualitatively, we show that the descriptions produced by LatentLens are semantically meaningful and provide more fine-grained interpretations for humans compared to individual tokens. More broadly, our findings contribute new evidence on the alignment between vision and language representations, opening up new directions for analyzing latent representations.

2026-04-29

International Conference on Machine Learning (accepted)

doi.org

openreview.net

MIRA: A Score for Conditional Distribution Accuracy and Model Comparison

Sammy Sharief

Justine Zeghal

Gabriel Missael Barco

Pablo Lemos

Yashar Hezaveh

Laurence Perreault-Levasseur

We present Mira, a method for estimating the expected probability that samples from a candidate conditional distribution match the true, unk… (see more)nown conditional distribution, for which only data-label pairs are available. We derive theoretical bounds obtained when the candidate distribution matches the true one and when the conditional distributions are independent. This framework thus enables model comparison by quantifying the alignment between the conditional distribution of a candidate model and the data-label pairs of the true model. Consequently, Mira enables Bayesian model comparison through direct posterior validation, bypassing the challenging evidence computation. We demonstrate its effectiveness across several toy problems and Bayesian inference tasks.

2026-04-29

International Conference on Machine Learning (spotlight)

doi.org

openreview.net

Operationalising the Superficial Alignment Hypothesis via Task Complexity

Tomás Vergara-Browne

Tiago Pimentel

The superficial alignment hypothesis (SAH) posits that large language models learn most of their knowledge during pre-training, and that pos… (see more)t-training merely surfaces this knowledge. The SAH, however, lacks a precise definition, which has led to (i) different and seemingly orthogonal arguments supporting it, and (ii) important critiques to it. We propose a new metric called task complexity: the length of the shortest program that achieves a target performance on a task. In this framework, the SAH simply claims that pre-trained models drastically reduce the complexity of achieving high performance on many tasks. Our definition unifies prior arguments supporting the SAH, interpreting them as different strategies to find such short programs. Experimentally, we estimate the task complexity of mathematical reasoning, machine translation, and instruction following; we then show that these complexities can be remarkably low when conditioned on a pre-trained model. Further, we find that pre-training enables access to strong performances on our tasks, but it can require programs of gigabytes of length to access them. Post-training, on the other hand, collapses the complexity of reaching this same performance by several orders of magnitude. Overall, our results highlight that task adaptation often requires surprisingly little information -- often just a few kilobytes.

2026-04-29

International Conference on Machine Learning (accepted)

doi.org

openreview.net

Overcoming the Modality Gap in Context-Aided Forecasting

Vincent Zhihao Zheng

Étienne Marcotte

Arjun Ashok

Andrew Robert Williams

Lijun Sun

Alexandre Drouin

Valentina Zantedeschi

Context-aided forecasting (CAF) holds promise for integrating domain knowledge and forward-looking information, enabling AI systems to surpa… (see more)ss traditional statistical methods. However, recent empirical studies reveal a puzzling gap: multimodal models often fail to outperform their unimodal counterparts. We hypothesize that this underperformance stems from poor context quality in existing datasets, as verification is challenging. To address these limitations, we introduce a semi-synthetic data augmentation method that generates contexts both descriptive of temporal dynamics and verifiably complementary to numerical histories. This approach enables massive-scale dataset creation, resulting in CAF-7M, a corpus of 7 million context-augmented time series windows, including a rigorously verified test set. We demonstrate that semi-synthetic pre-training transfers effectively to real-world evaluation, and show clear evidence of context utilization. Our results suggest that dataset quality, rather than architectural limitations, has been the primary bottleneck in context-aided forecasting.

2026-04-29

International Conference on Machine Learning (accepted)

doi.org

openreview.net

Mila's Community of Practice: Agentic AI

Venture Sprint

AI Advantage: Productivity in Public Service

Publications

Mila's Community of Practice: Agentic AI

Venture Sprint

AI Advantage: Productivity in Public Service

Popular keywords:

Publications