Publications

The Superposition of Diffusion Models Using the Itô Density Estimator
Marta Skreta
Lazar Atanackovic
Avishek Joey Bose
Alexander Tong
Kirill Neklyudov
The Cambrian explosion of easily accessible pre-trained diffusion models suggests a demand for methods that combine multiple different pre-t… (see more)rained diffusion models without incurring the significant computational burden of re-training a larger combined model. In this paper, we cast the problem of combining multiple pre-trained diffusion models at the generation stage under a novel proposed framework termed superposition. Theoretically, we derive superposition from rigorous first principles stemming from the celebrated continuity equation and design two novel algorithms tailor-made for combining diffusion models in SuperDiff. SuperDiff leverages a new scalable Itô density estimator for the log likelihood of the diffusion SDE which incurs no additional overhead compared to the well-known Hutchinson's estimator needed for divergence calculations. We demonstrate that SuperDiff is scalable to large pre-trained diffusion models as superposition is performed solely through composition during inference, and also enjoys painless implementation as it combines different pre-trained vector fields through an automated re-weighting scheme. Notably, we show that SuperDiff is efficient during inference time, and mimics traditional composition operators such as the logical OR and the logical AND. We empirically demonstrate the utility of using SuperDiff for generating more diverse images on CIFAR-10, more faithful prompt conditioned image editing using Stable Diffusion, as well as improved conditional molecule generation and unconditional de novo structure design of proteins. https://github.com/necludov/super-diffusion
2025-04-22
International Conference on Learning Representations (Accept (Spotlight))
doi.org
openreview.net
Towards Improving Exploration Through Sibling Augmented GFlowNets
Kanika Madan
Alex Lamb
Emmanuel Bengio
Glen Berseth
Yoshua Bengio
2025-04-22
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning
Khimya Khetarpal
Zhaohan Daniel Guo
Bernardo Avila Pires
Yunhao Tang
Clare Lyle
Mark Rowland
Nicolas Heess
Diana Borsa
Arthur Guez
Will Dabney
2025-04-22
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics (published)
doi.org
proceedings.mlr.press
VCR: A Task for Pixel-Level Complex Reasoning in Vision Language Models via Restoring Occluded Text
Tianyu Zhang
Suyuchen Wang
Lu Li
Ge Zhang
Perouz Taslakian
Sai Rajeswar
Jie Fu
Bang Liu
Yoshua Bengio
We introduce Visual Caption Restoration (VCR), a novel vision-language task that challenges models to accurately restore partially obscured … (see more)texts using pixel-level hints within images. This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images. While numerous works have integrated text embedded in images into visual question-answering tasks, approaches to these tasks generally rely on optical character recognition or masked language modeling, thus reducing the task to mainly text-based processing. However, text-based processing becomes ineffective in VCR as accurate text restoration depends on the combined information from provided images, context, and subtle cues from the tiny exposed areas of masked texts. We develop a pipeline to generate synthetic images for the VCR task using image-caption pairs, with adjustable caption visibility to control the task difficulty. With this pipeline, we construct a dataset for VCR called VCR-Wiki using images with captions from Wikipedia, comprising 2.11M English and 346K Chinese entities in both easy and hard split variants. Our results reveal that current vision language models significantly lag behind human performance in the VCR task, and merely fine-tuning the models on our dataset does not lead to notable improvements. We release VCR-Wiki and the data construction code to facilitate future research.
2025-04-22
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models
Andrea Tirinzoni
Ahmed Touati
Jesse Farebrother
Mateusz Guzek
Anssi Kanervisto
Yingchen Xu
Alessandro Lazaric
Matteo Pirotta
Unsupervised reinforcement learning (RL) aims at pre-training models that can solve a wide range of downstream tasks in complex environments… (see more). Despite recent advancements, existing approaches suffer from several limitations: they may require running an RL process on each task to achieve a satisfactory performance, they may need access to datasets with good coverage or well-curated task-specific samples, or they may pre-train policies with unsupervised losses that are poorly correlated with the downstream tasks of interest. In this paper, we introduce FB-CPR, which regularizes unsupervised zero-shot RL based on the forward-backward (FB) method towards imitating trajectories from unlabeled behaviors. The resulting models learn useful policies imitating the behaviors in the dataset, while retaining zero-shot generalization capabilities. We demonstrate the effectiveness of FB-CPR in a challenging humanoid control problem. Training FB-CPR online with observation-only motion capture datasets, we obtain the first humanoid behavioral foundation model that can be prompted to solve a variety of whole-body tasks, including motion tracking, goal reaching, and reward optimization. The resulting model is capable of expressing human-like behaviors and it achieves competitive performance with task-specific methods while outperforming state-of-the-art unsupervised RL and model-based baselines.
2025-04-22
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
ZETA: Leveraging $Z$-order Curves for Efficient Top-$k$ Attention
Qiuhao Zeng
Jerry Huang
Peng Lu
Gezheng Xu
Boxing Chen
Charles Ling
Boyu Wang
Over recent years, the Transformer has become a fundamental building block for sequence modeling architectures. Yet at its core is the use o… (see more)f self-attention, whose memory and computational cost grow quadratically with the sequence length
2025-04-22
International Conference on Learning Representations (Accept (Poster))
openreview.net
openreview.net
LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilities
Thomas Schmied
Jörg Bornschein
Jordi Grau-Moya
Markus Wulfmeier
Razvan Pascanu
2025-04-21
ArXiv (preprint)
doi.org
arxiv.org
Neural Kinematic Bases for Fluids
Yibo Liu
Paul Kry
Kenny Erleben
Noam Aigerman
Sune Darkner
Teseo Schneider
2025-04-21
ArXiv (preprint)
doi.org
arxiv.org
Cortical differences across psychiatric disorders and associated common and rare genetic variants
Kuldeep Kumar
Zhijie Liao
Jakub Kopal
Clara Moreau
Christopher R. K. Ching
Claudia Modenato
Will Snyder
Sayeh Kazem
Charles-Olivier Martin
C.O. Martin
Anne-Marie Bélanger
Valérie K. Fontaine
Khadije Jizi
Rune Boen
Guillaume Huguet
Zohra Saci
Leila Kushan
Ana I. Silva
Marianne B.M. van den Bree
David E.J. Linden … (see 16 more)
Michael J. Owen
Jeremy Hall
Sarah Lippé
Guillaume Dumas
Bogdan Draganski
Laura Almasy
Sophia I. Thomopoulos
Neda Jahanshad
Ida E. Sønderby
Ole A. Andreassen
David C. Glahn
Armin Raznahan
Carrie Bearden
Tomáš Paus
Paul M. Thompson
Sébastien Jacquemont
2025-04-18
medRxiv (preprint)
doi.org
Deep Learning Unlocks the True Potential of Organ Donation after Circulatory Death with Accurate Prediction of Time-to-Death
Xingzhi Sun
Edward De Brouwer
Chen Liu
Smita Krishnaswamy
Ramesh Batra
𝟏
Increasing the number of organ donations after circulatory death (DCD) has been identified as one of the most important ways of addressing t… (see more)he ongoing organ shortage. While recent technological advances in organ transplantation have increased their success rate, a substantial challenge in increasing the number of DCD donations resides in the uncertainty regarding the timing of cardiac death after terminal extubation, impacting the risk of prolonged ischemic organ injury, and negatively affecting post-transplant outcomes. In this study, we trained and externally validated an ODE-RNN model, which combines recurrent neural network with neural ordinary equations and excels in processing irregularly-sampled time series data. The model is designed to predict time-to-death following terminal extubation in the intensive care unit (ICU) using the last 24 hours of clinical observations. Our model was trained on a cohort of 3,238 patients from Yale New Haven Hospital, and validated on an external cohort of 1,908 patients from six hospitals across Connecticut. The model achieved accuracies of 95.3 {+/-} 1.0% and 95.4 {+/-} 0.7% for predicting whether death would occur in the first 30 and 60 minutes, respectively, with a calibration error of 0.024 {+/-} 0.009. Heart rate, respiratory rate, mean arterial blood pressure (MAP), oxygen saturation (SpO2), and Glasgow Coma Scale (GCS) scores were identified as the most important predictors. Surpassing existing clinical scores, our model sets the stage for reduced organ acquisition costs and improved post-transplant outcomes.
2025-04-18
Scientific Reports (published)
doi.org
Impact of Reducing Time Lived With Colostomies on Social Stigma Affecting Children With Anorectal Malformations in Southwestern Uganda.
Felix Oyania
Caroline Q. Stephens
Sarah Ullrich
Meera Kotagal
Amy M. Shui
Caleb Tuhumwire
Godfrey Zari Rukundo
Joseph Ngonzi
Ava Yap
Francis Bajunirwe
Dan Poenaru
Doruk Ozgediz
BACKGROUND The social stigma of families of children living with colostomies due to anorectal malformation (ARM) is significant in low-incom… (see more)e countries (LICs). Improved access to pediatric surgery has resulted in more 1-stage ARM procedures in Southwestern Uganda, avoiding colostomy creation, but the impact on social stigma experienced by families is unknown. We hypothesized that this change would decrease the social stigma experienced by families. METHODS A single-center mixed retrospective and prospective cohort study with combined qualitative data of families of children with ARM who underwent corrective surgery compared the stigma experienced by those with colostomies to those without. The Kilifi Stigma Scale of Epilepsy (KSSE) was used to assess social stigma. Multivariable regression analysis assessed differences in the stigma experienced, controlling for age at diagnosis, rurality, distance traveled, sex, and parental education. Subgroup analysis assessed the impact of colostomy duration on stigma, stratified over parental education. RESULTS Patient/family dyads with 238 ARM were included; 177 (74%) received a colostomy. Most patients were male (51%), lived in rural areas (71%), and had parents with primary school education (65%). For those without a colostomy, the median KSSE was 0 (Q1-Q3 0-0), compared to 11 (Q1-Q3 3-20) for colostomy. On multivariable analysis, after controlling for age at diagnosis, rurality, distance traveled, sex, and parental education attainment, families of patients with ARM who received a colostomy had a median KSSE score 7.8 points higher than those who did not receive a colostomy (coefficient 7.78, 95% 3.14-12.43, and p = 0.001). When the duration of colostomy (in years) was examined, the median KSSE score increased by 1.58 points for each additional year for a patient who had a colostomy (IRR 1.58, 95% CI: 0.76-2.40, and p  0.001). CONCLUSION Adopting a 1-stage ARM repair for the select types, which avoids colostomy creation, significantly reduces the exper
2025-04-18
World Journal of Surgery (published)
doi.org
Learning from Stochastic Teacher Representations Using Student-Guided Knowledge Distillation
Muhammad Haseeb Aslam
Clara Martinez
Marco Pedersoli
Alessandro Lameiras Koerich
Ali Etemad
Éric Granger
Advances in self-distillation have shown that when knowledge is distilled from a teacher to a student using the same deep learning (DL) arch… (see more)itecture, the student performance can surpass the teacher particularly when the network is overparameterized and the teacher is trained with early stopping. Alternatively, ensemble learning also improves performance, although training, storing, and deploying multiple models becomes impractical as the number of models grows. Even distilling an ensemble to a single student model or weight averaging methods first requires training of multiple teacher models and does not fully leverage the inherent stochasticity for generating and distilling diversity in DL models. These constraints are particularly prohibitive in resource-constrained or latency-sensitive applications such as wearable devices. This paper proposes to train only one model and generate multiple diverse teacher representations using distillation-time dropout. However, generating these representations stochastically leads to noisy representations that are misaligned with the learned task. To overcome this problem, a novel stochastic self-distillation (SSD) training strategy is introduced for filtering and weighting teacher representation to distill from task-relevant representations only, using student-guided knowledge distillation (SGKD). The student representation at each distillation step is used as authority to guide the distillation process. Experimental results on real-world affective computing, wearable/biosignal datasets from the UCR Archive, the HAR dataset, and image classification datasets show that the proposed SSD method can outperform state-of-the-art methods without increasing the model size at both training and testing time, and incurs negligible computational complexity compared to state-of-the-art ensemble learning and weight averaging methods.
2025-04-18
ArXiv (preprint)
doi.org
arxiv.org