Publications

$\mu$LO: Compute-Efficient Meta-Generalization of Learned Optimizers
Benjamin Therien
Charles-Etienne Joseph
Boris Knyazev
Edouard Oyallon
Irina Rish
Eugene Belilovsky
Learned optimizers (LOs) have the potential to significantly reduce the wall-clock training time of neural networks. However, they can strug… (voir plus)gle to optimize unseen tasks (*meta-generalize*), especially when training networks wider than those seen during meta-training. To address this, we derive the Maximal Update Parametrization (
2025-12-31
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
Conditionally Site-Independent Neural Evolution of Antibody Sequences
Stephen Zhewen Lu
Aakarsh Vermani
Kohei Sanno
Jiarui Lu
Frederick A. Matsen IV
Milind Jagota
Yun S. Song
Common deep learning approaches for antibody engineering focus on modeling the marginal distribution of sequences. By treating sequences as … (voir plus)independent samples, however, these methods overlook affinity maturation as a rich and largely untapped source of information about the evolutionary process by which antibodies explore the underlying fitness landscape. In contrast, classical phylogenetic models explicitly represent evolutionary dynamics but lack the expressivity to capture complex epistatic interactions. We bridge this gap with **CoSiNE**, a continuous-time Markov chain parameterized by a deep neural network. Mathematically, we prove that CoSiNE provides a first-order approximation to the intractable sequential point mutation process, capturing epistatic effects with an error bound that is quadratic in branch length. Empirically, CoSiNE outperforms state-of-the-art language models in zero-shot variant effect prediction by explicitly disentangling selection from context-dependent somatic hypermutation. Finally, we introduce *Guided Gillespie*, a classifier-guided sampling scheme that steers CoSiNE at inference time, enabling efficient optimization of antibody binding affinity toward specific antigens.
2025-12-31
International Conference on Machine Learning (Accept (regular))
doi.org
openreview.net
Contractive Diffusion Policies: Robust Action Diffusion via Contractive Score-Based Sampling with Differential Equations
Amin Abyaneh
Charlotte Morissette
Mohamad H. Danesh
Anas El Houssaini
David Meger
Gregory Dudek
Hsiu-Chin Lin
Diffusion policies have emerged as powerful generative models for offline policy learning, whose sampling process can be rigorously characte… (voir plus)rized by a score function guiding a Stochastic Differential Equation (SDE). However, the same score-based SDE modeling that grants diffusion policies the flexibility to learn diverse behavior also incurs solver and score-matching errors, large data requirements, and inconsistencies in action generation. While less critical in image generation, these inaccuracies compound and lead to failure in continuous control settings. We introduce Contractive Diffusion Policies (CDPs) to induce contractive behavior in the diffusion sampling dynamics. Contraction pulls nearby flows closer to enhance robustness against solver and score-matching errors while reducing unwanted action variance. We develop an in-depth theoretical analysis along with a practical implementation recipe to incorporate CDPs into existing diffusion policy architectures with minimal modification and computational cost. We evaluate CDPs for offline learning by conducting extensive experiments in simulation and real-world settings. Across benchmarks, CDPs often outperform baseline policies, with pronounced benefits under data scarcity.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
ControBench: An Interaction-Aware Benchmark for Controversial Discourse Analysis on Social Networks
Ta Thanh Thuy
Jiaqi Zhu
Xuan Liu
Lin Shang
Reihaneh Rabbany
Guillaume Rabusseau
Lihui Chen
Zheng Yilun
Sitao Luan
Understanding how people argue across ideological divides online is important for studying political polarization, misinformation, and conte… (voir plus)nt moderation. Existing datasets capture only part of this problem: some preserve text but ignore interaction structure, some model structure without rich semantics, and others represent conversations without stable user-level ideological identity. We introduce ControBench, a benchmark for controversial discourse analysis that combines heterogeneous social interaction graphs with rich textual semantics. Built from Reddit discussions on three topics, Trump, abortion, and religion, ControBench contains 7,370 users, 1,783 posts, and 26,525 interactions. The graph contains user and post nodes connected by semantically enriched edges; in particular, user-comment-user edges encode both a reply and the parent comment that it responds to, preserving local argumentative context. User labels are derived from self-declared Reddit flairs, providing a scalable proxy for ideological identity without manual annotation. The resulting datasets exhibit low or negative adjusted homophily (Trump: -0.77, Abortion: 0.06, Religion: 0.04), reflecting the cross-cutting structure of real-world debate. We evaluate graph neural networks, pretrained language models, and large language models on ControBench and observe distinct performance patterns across topics and model families, especially when ideological boundaries are ambiguous. These results position ControBench as a challenging and realistic benchmark for controversial discourse analysis.
2025-12-31
Unpublished (publié)
doi.org
arxiv.org
Decoding Dynamic Visual Experience from Calcium Imaging via Cell-Pattern-Aware SSL
Sangyoon Bae
Mehdi Azabou
Blake A Richards
Jiook Cha
Self-supervised learning (SSL) holds a great deal of promise for applications in neuroscience, due to the lack of large-scale, consistently … (voir plus)labeled neural datasets. However, most neural datasets contain heterogeneous populations that mix stable, predictable cells with highly stochastic, stimulus-contingent ones, which has made it hard to identify consistent activity patterns during SSL. As a result, self-supervised pretraining has yet to show clear signs of benefits from scale on neural data. Here, we present a novel approach to self-supervised pretraining, POYO-SSL that exploits the heterogeneity of neural data to improve pretraining and achieve benefits of scale. Specifically, in POYO-SSL we pretrain only on predictable neurons---identified on the pretraining split via simple higher-order statistics (skewness and kurtosis)---then we fine-tune on the unpredictable population for downstream tasks. On the Allen Brain Observatory dataset, this strategy yields approximately 12--13\% relative gains over from-scratch training and exhibits smooth, monotonic scaling with model size. In contrast, existing state-of-the-art baselines plateau or destabilize as model size increases. By making predictability an explicit metric for crafting the data diet, POYO-SSL turns heterogeneity from a liability into an asset, providing a robust, biologically grounded recipe for scalable neural decoding and a path toward foundation models of neural dynamics.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
A Deep Learning and Inertia-Aware Load Shedding Framework for Mitigating Load-Altering Attacks
Anoosh Dini
Keyhan Sheshyekani
Hanane Dagdougui
The widespread integration of information and communication technologies into modern power systems has increased their vulnerability to cybe… (voir plus)r-physical threats, such as load-altering attacks (LAA). These attacks can cause rapid load changes, potentially triggering protective mechanisms like under-frequency load shedding (UFLS). Existing approaches for mitigating these attacks are limited, and they mostly rely on preventive measures or neglect system dynamics. In this paper, we propose a novel online framework for the detection and mitigation of LAAs that addresses these limitations. The detection component employs a convolutional neural network–long short-term memory autoencoder (CNN-LSTM AE) architecture to capture anomalies in load consumption data. For mitigation, we propose an inertia-aware load shedding scheme that dynamically adjusts the shedding amount based on the real-time frequency and the magnitude of the attack. This approach prevents overshedding caused by predefined UFLS relay settings and mitigates undershedding by considering the system’s real-time inertia. To this end, a variable forgetting factor recursive least squares (VFF-RLS) algorithm is proposed, which can track inertia variations within a few seconds. The proposed framework is compatible with both synchronous generator-based and converter-interfaced generator-dominated grids. Simulations indicate the effectiveness of the proposed framework in maintaining frequency stability under a wide range of attack scenarios.
2025-12-31
IEEE Access (publié)
doi.org
Deep neural networks divide and conquer dihedral multiplication
Sihui Wei
Gavin McCracken
Gabriela Moisescu-Pareja
Harley Wiltzer
Doina Precup
Irina Rish
Jonathan Love
We find multilayer perceptrons and transformers both universally learn an instantiation of the same divide-and-conquer algorithm that requir… (voir plus)es only a logarithmic number of neural representations to solve dihedral multiplication. Clustering neurons based on similar activation behaviour reveals remarkably clear structure: each neural representation corresponds to a Cayley graph. To our knowledge, this is the first work that fully characterizes and describes all neural representations that are learnable on a dataset, while prior work on group multiplications studied neuron-level behavior, or preliminarily investigated cluster behavior. Thus, we can understand the algorithm networks universally learn at three levels of abstraction: 1) Neurons activate on coset or approximate coset structure of the dihedral group. 2) Groups of neurons together form neural representations that act to divide the dataset into different subproblems, being Cayley graphs, where the equivalence class of the answer is computed. 3) The global algorithm then linearly combines each neural representation (subproblem) together at the logits. This work provides a deep case study and provides the community with a very well understood toy model for interpretability, as well as makes steps toward proving the conjecture that DNNs will divide and conquer all group multiplication tasks.
2025-12-31
International Conference on Machine Learning (Accept (regular))
openreview.net
openreview.net
Development and Deployment of Jami: Experiences with a Mobile Distributed Communication Platform
Yili Jin
Tianyi Yang
Adrien Béraud
Tao Yang
Sébastien Blin
Cyrille Béraud
Qiao Xiang
Xue Liu
2025-12-31
IEEE Transactions on Mobile Computing (publié)
doi.org
Diffusion Alignment as Variataional Expectation-Maximization
Jaewoo Lee
Minsu Kim
Sanghyeok Choi
Inhyuck Song
Sujin yun
Hyeongyu Kang
Woocheol Shin
Taeyoung YUN
Kiyoung Om
Jinkyoo Park
Diffusion alignment aims to optimize diffusion models for the downstream objective. While existing methods based on reinforcement learning o… (voir plus)r direct backpropagation achieve considerable success in maximizing rewards, they often suffer from reward over-optimization and mode collapse. We introduce Diffusion Alignment as Variational Expectation-Maximization (DAV), a framework that formulates diffusion alignment as an iterative process alternating between two complementary phases: the E-step and the M-step. In the E-step, we employ test-time search to generate diverse and reward-aligned samples. In the M-step, we refine the diffusion model using samples discovered by the E-step. We demonstrate that DAV can optimize reward while preserving diversity for both continuous and discrete tasks: text-to-image synthesis and DNA sequence design.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
Discovering Diverse Behaviors via Temporal Contrastive Learning
Faisal Mohamed
Catherine Ji
Benjamin Eysenbach
Glen Berseth
Effective exploration in reinforcement learning requires not only tracking where an agent has been, but also understanding how the agent per… (voir plus)ceives and represents the world. To learn powerful representations, an agent should actively explore states that contribute to its knowledge of the environment. Temporal representations can capture the information necessary to solve a wide range of potential tasks while avoiding the computational cost associated with full state reconstruction. In this paper, we propose an exploration method that leverages temporal contrastive representations to guide exploration, prioritizing states with unpredictable future outcomes. We demonstrate that such representations can enable the learning of complex exploratory behaviors in locomotion, manipulation, and embodied-AI tasks, revealing capabilities and behaviors that traditionally require extrinsic rewards. Unlike approaches that rely on explicit distance learning or episodic memory mechanisms (e.g., quasimetric-based methods), our method builds directly on temporal similarities, yielding a simpler yet effective strategy for exploration.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
DisTaC: Conditioning Task Vectors via Distillation for Robust Model Merging
Kotaro Yoshida
Yuji Naraki
Takafumi Horie
Ryotaro Shimizu
Ioannis Mitliagkas
Hiroki Naganuma
Model merging has emerged as an efficient and flexible paradigm for multi-task learning, with numerous methods being proposed in recent year… (voir plus)s. However, these state-of-the-art techniques are typically evaluated on benchmark suites that are highly favorable to model merging, and their robustness in more realistic settings remains largely unexplored. In this work, we first investigate the vulnerabilities of model-merging methods and pinpoint the source-model characteristics that critically underlie them. Specifically, we identify two factors that are particularly harmful to the merging process: (1) disparities in task vector norms, and (2) the low confidence of the source models. To address this issue, we propose **DisTaC** (**Dis**tillation for **Ta**sk vector **C**onditioning), a novel method that pre-conditions these problematic task vectors before the merge. DisTaC leverages knowledge distillation to adjust a task vector's norm and increase source-model confidence while preserving its essential task-specific knowledge. Our extensive experiments demonstrate that by pre-conditioning task vectors with DisTaC, state-of-the-art merging techniques can successfully integrate models that exhibit these harmful traits, where they would otherwise fail, and achieve significant performance gains.
2025-12-31
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
DSL: Dual-Stage List Decoding for Polar Turbo Product Coded Massive MIMO Towards 6 G Extreme Connectivity
Jian Zheng
Yu Tian
Huayi Zhou
Xiaosi Tan
Yutai Sun
Warren J. Gross
Xiaohu You
Chuan Zhang
Targeting sixth generation (6 G) extreme connectivity, massive multiple-input multiple-output (mMIMO) systems face challenges in achieving b… (voir plus)oth high data rates and low latency. Polar turbo product codes (polar-TPCs) with near-optimal error correction performance under constrained latency are promising for 6 G mMIMO applications. In this letter, we propose a dual-stage list (DSL) decoding scheme for polar-TPC coded mMIMO systems, leveraging a turbo list decoder followed by a tree-search-based post-processing method to enhance receiver performance. Moreover, we introduce a two-phase codeword validity check strategy to address the complexity challenges. Empirical results demonstrate that the receiver with our proposed DSL decoding outperforms existing receivers, achieving a superior trade-off between performance and complexity.
2025-12-31
IEEE Transactions on Vehicular Technology (publié)
doi.org