Publications

Training Diffusion Language Models for Black-Box Optimization
Zipeng Sun
Can (Sam) Chen
Ye Yuan
Haolun Wu
Jiayao Gu
Chris Pal
Xue Liu
We study offline black-box optimization (BBO), aiming to discover improved designs from an offline dataset of designs and labels, a problem … (see more)common in robotics, DNA, and materials science with limited labeled samples. While recent work applies autoregressive LLMs to BBO by formatting tasks as natural-language prompts, their left-to-right design generation struggles to capture the strong bidirectional dependencies inherent in design problems. To address this, we propose adapting diffusion LLMs to offline BBO to leverage their bidirectional modeling capabilities. However, a domain gap exists between the natural text pre-training of diffusion LLMs and the heterogeneous signals in BBO (prompts, designs, and labels). To bridge this gap, we construct a unified prompt–response corpus and introduce delimiter tokens to explicitly mark field boundaries for *domain adaptation*. We further propose a two-stage *post-training* framework to align the diffusion LLM generation with high-label designs. The first stage performs supervised fine-tuning on the unified dataset via masked-response prediction, and the second stage adopts reinforcement learning with rewards defined by label improvements. Our method achieves state-of-the-art results on Design-Bench small-data settings. Code for our work is available here: https://anonymous.4open.science/r/Anonymous-dllm4bbo-D78A/README.md.
2026-04-29
International Conference on Machine Learning (spotlight)
doi.org
openreview.net
DreamProver: Evolving Transferable Lemma Libraries via a Wake-Sleep Theorem-Proving Agent
Youyuan Zhang
Jialiang Sun
Hangrui Bi
Chuqin Geng
Wenjie Ma
Zhaoyu Li
Xujie Si
We introduce DreamProver, an agentic framework that leverages a "wake-sleep" program induction paradigm to discover reusable lemmas for form… (see more)al theorem proving. Existing approaches either rely on fixed lemma libraries, which limit adaptability, or synthesize highly specific intermediate lemmas tailored to individual theorems, thereby lacking generality. DreamProver addresses this gap through an iterative two-stage process. In the wake stage, DreamProver attempts to prove theorems from a training set using the current lemma library while proposing new candidate lemmas. In the "sleep" stage, it abstracts, refines, and consolidates these candidates to compress and optimize the library. Through this alternating cycle, DreamProver progressively evolves a compact set of high-level, transferable lemmas that can be effectively used to prove unseen theorems in related domains. Experimental results demonstrate that DreamProver substantially improves proof success rates across a diverse set of mathematical benchmarks, while also producing more concise proofs and reducing computational cost.
2026-04-28
arXiv (preprint)
doi.org
arxiv.org
Split over $n$ resource sharing problem: Are fewer capable agents better than many simpler ones?
Karthik Soma
Mohamed S. Talamali
Genki Miyauchi
Giovanni Beltrame
Heiko Hamann
Roderich Groß
In multi-agent systems, should limited resources be concentrated into a few capable agents or distributed among many simpler ones? This work… (see more) formulates the split over
2026-04-28
arXiv (preprint)
doi.org
arxiv.org
When 2D Tasks Meet 1D Serialization: On Serialization Friction in Structured Tasks
C.-T.D. Lo
Lu Li
Diji Yang
Tianyu Zhang
Yunkai Zhang
Yoshua Bengio
Yi Zhang
Chung-Hsiang Lo
In the LLM era, many symbolic and structured problems are presented to models through 1D text serialization. Yet some such problems are nati… (see more)vely two-dimensional: their relevant relations, such as row--column correspondence or spatial adjacency, are defined by position in a 2D layout rather than by sequential order. This raises a representational question: does preserving the same symbolic entries in a 1D sequence also preserve the relational structure needed for computation? We study this issue through the lens of serialization friction: the representational mismatch in which the same underlying task instances and entries are still present, but relations that depend on layout become implicit under 1D serialization. The study uses a controlled synthetic testbed of three tasks: matrix transpose, Conway's Game of Life, and LU decomposition. In each task, the same instances are presented either as 1D text serialization or as their native 2D layout rendered as an image. Across this testbed, 1D serialization degrades more sharply as task size grows, and errors under serialization exhibit spatially structured patterns, suggesting that this presentation choice is consequential within our testbed. To further interpret these results, we add supplementary analyses that include a within-visual probe and an additional comparison of the two input presentations under the mixed-training transpose setting. These findings suggest that, for layout-defined tasks, reducing inputs to 1D serialization is not a neutral choice of representation.
2026-04-28
arXiv (preprint)
doi.org
arxiv.org
Learning Structure, Energy, and Dynamics: A Survey of Artificial Intelligence for Protein Dynamics
Haocheng Tang
Liang Shi
Ya-Shi Zhang
Xixian Liu
Jian Tang
Jiarui Lu
Protein dynamics underlie many biological functions, yet remain difficult to characterize due to the high computational cost of molecular dy… (see more)namics simulations and the scarcity of dynamic structural data. This survey reviews recent advances in artificial intelligence for protein dynamics from three perspectives: learning from structural ensembles and trajectories, learning from physical energy signals, and learning to accelerate molecular simulations. We summarize representative methods for conformation ensemble generation, trajectory generation, Boltzmann generators, physics-aware adaptation, machine learning potentials, coarse-grained modeling, and collective variable discovery. We further discuss available datasets and key open challenges, such as scalability, thermodynamic consistency, kinetic fidelity, and integration with experimental constraints.
2026-04-27
arXiv (preprint)
doi.org
arxiv.org
Seasonal Peaks and Climatic Predictors of Chronic Urticaria: A Global Google Trends Analysis
Katya Peri
Connor Prosty
Han Zhang Huang
Catherine Silotch
Gazal Javed
Maxine Joly-Chevrier
Moshe Ben-Shoshan
Elham Rahme
Ivan V Litvinov
Maria Cutumisu
Qiuyan Yu
Elena Netchiporouk
2026-04-27
Journal of Cutaneous Medicine and Surgery (published)
doi.org
The Cost of Expertise: Understanding MoE Decode Performance
Sami Abuzakuk
Oana Balmau
Jiaxuan Chen
Anne-Marie Kermarrec
Rafael Pires
Ramya Prabhu
Martijn de Vos
2026-04-27
EuroMLSys @ European Conference on Computer Systems (published)
doi.org
The Role of Symmetry in Optimizing Overparameterized Networks
Kusha Sareen
Mohammad Pedramfar
Sékou-Oumar Kaba
Mehran Shakerinava
Siamak Ravanbakhsh
Overparameterization is central to the success of deep learning, yet the mechanisms by which it improves optimization remain incompletely un… (see more)derstood. We analyze weight-space symmetries in neural networks and show that overparameterization introduces additional symmetries that benefit optimization in two distinct ways. First, we prove that these symmetries act as a form of diagonal preconditioning on the Hessian, enabling the existence of better-conditioned minima within each equivalence class of functionally identical solutions. Second, we show that overparameterization increases the probability mass of global minima near typical initializations, making these favorable solutions more reachable. Teacher-student network experiments validate our theoretical predictions: as width increases, the Hessian trace decreases, condition numbers improve, and convergence accelerates. Our analysis provides a unified framework for understanding overparameterization and width growth as a geometric transformation of the loss landscape.
2026-04-27
arXiv (preprint)
doi.org
arxiv.org
Adaptive Prompt Embedding Optimization for LLM Jailbreaking
Miles Q. Li
Benjamin C. M. Fung
Boyang Li
Radin Hamidi Rad
Ebrahim Bagheri
Existing white-box jailbreak attacks against aligned LLMs typically append discrete adversarial suffixes to the user prompt, which visibly a… (see more)lters the prompt and operates in a combinatorial token space. Prior work has avoided directly optimizing the embeddings of the original prompt tokens, presumably because perturbing them risks destroying the prompt's semantic content. We propose Prompt Embedding Optimization (PEO), a multi-round white-box jailbreak that directly optimizes the embeddings of the original prompt tokens without appending any adversarial tokens, and show that the concern is unfounded: the optimized embeddings remain close enough to their originals that the visible prompt string is preserved exactly after nearest-token projection, and quantitative analysis shows the model's responses stay on topic for the large majority of prompts. PEO combines continuous embedding-space optimization with structured continuation targets and an adaptive failure-focused schedule. Counterintuitively, later PEO rounds can benefit from heuristic composite response scaffolds that are not natural standalone templates, yet ASR-Judge shows that the resulting gains are not merely empty formatting or scaffold-only outputs. Across two standard harmful-behavior benchmarks and competing white-box attacks spanning discrete suffix search, appended adversarial embeddings, and search-based adversarial generation, PEO outperforms all of them in our experiments.
2026-04-26
arXiv (preprint)
doi.org
arxiv.org
Hierarchical Behaviour Spaces
Michael Matthews
Anssi Kanervisto
Jakob Foerster
Pierluca D’Oro
Scott Fujimoto
Mikael Henaff
Recent work in hierarchical reinforcement learning has shown success in scaling to billions of timesteps when learning over a set of predefi… (see more)ned option reward functions. We show that, instead of using a single reward function per option, the reward functions can be effectively used to induce a space of behaviours, by letting the controller specify linear combinations over reward functions, allowing a more expressive set of policies to be represented. We call this method Hierarchical Behaviour Spaces (HBS). We evaluate HBS on the NetHack Learning Environment, demonstrating strong performance. We conduct a series of experiments and determine that, perhaps going against conventional wisdom, the benefits of hierarchy in our method come from increased exploration rather than long term reasoning.
2026-04-26
arXiv (preprint)
doi.org
arxiv.org
Magnetic phases of the anisotropic triangular Hubbard model from the ghost-Gutzwiller approximation in the rotating spin-frame
Azin Kazemi-Moridani
Samuele Giuli
Tsung-Han Lee
A. -M. S. Tremblay
Michel Côté
Nicola Lanatà
Olivier Gingras
We investigate the magnetic phase diagram of the half-filled Hubbard model on the anisotropic triangular lattice using the Gutzwiller approx… (see more)imation (GA) and its ghost generalization (ghost-GA). By combining a rotating spin-frame formulation with high-resolution momentum grids, we determine magnetic ground states through direct total-energy minimization over the ordering wavevector. We benchmark standard GA and ghost-GA against dynamical mean-field theory (DMFT) and dual-fermion results. We show that GA already captures the qualitative structure of the phase diagram, but systematically overestimates the stability of magnetic order due to the absence of dynamical fluctuations. We find that introducing a small number of auxiliary ''ghost'' orbitals is sufficient to recover most dynamical effects and significantly improves quantitative agreement with DMFT. Exploring the full Brillouin zone, we obtain a phase diagram comprising paramagnetic and various magnetic phases. In contrast to ladder dual-fermion susceptibility-based predictions, we find that the one-dimensional antiferromagnetic phase is never stabilized, despite being the leading instability in certain regimes. Our results establish ghost-GA as an efficient and systematically improvable framework for studying magnetism in frustrated systems, capable of achieving near-DMFT accuracy at a fraction of the computational cost. They also highlight that standard GA performs qualitatively well for capturing the general phase diagram, enabling the investigation of incommensurate magnetic orders in more complex systems.
2026-04-26
arXiv (unknown)
doi.org
arxiv.org
Scaling Properties of Continuous Diffusion Spoken Language Models
Jason Ramapuram
Eeshan Gunesh Dhekane
Amitis Shidani
Dan Busbridge
Bogdan Mazoure
Zijin Gu
Russ Webb
Tatiana Likhomanenko
Navdeep Jaitly
Speech-only spoken language models (SLMs) lag behind text and text-speech models in performance, with recent discrete autoregressive (AR) SL… (see more)Ms indicating significant computational and data demands to match text models. Since discretizing continuous speech for AR creates bottlenecks, we explore whether continuous diffusion (CD) SLM is more viable. To quantify the SLMs linguistic quality, we introduce the phoneme Jensen-Shannon divergence (pJSD) metric. Our analysis reveals CD SLMs, mirroring AR behavior, exhibit scaling laws for validation loss and pJSD, and show optimal token-to-parameter ratios decreasing as compute scales. However, for the latter, loss becomes insensitive to choice of data and model sizes, showing potential for fast inference. Scaling CD SLMs to 16B parameters with tens of millions of hours of conversational data enables generation of emotive, prosodic, multi-speaker, multilingual speech, though achieving long-form coherence remains a significant challenge.
2026-04-26
arXiv (preprint)
doi.org
arxiv.org