Publications

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
Rambod Azimi
Rishav Rishav
Marek Teichmann
Samira Ebrahimi Kahou
2024-10-28
ArXiv (preprint)
doi.org
arxiv.org
Trajectory Flow Matching with Applications to Clinical Time Series Modeling
Xi Zhang
Yuan Pu
Yuki Kawamura
Andrew Loza
Yoshua Bengio
Dennis Shung
Alexander Tong
Modeling stochastic and irregularly sampled time series is a challenging problem found in a wide range of applications, especially in medici… (see more)ne. Neural stochastic differential equations (Neural SDEs) are an attractive modeling technique for this problem, which parameterize the drift and diffusion terms of an SDE with neural networks. However, current algorithms for training Neural SDEs require backpropagation through the SDE dynamics, greatly limiting their scalability and stability. To address this, we propose Trajectory Flow Matching (TFM), which trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics. TFM leverages the flow matching technique from generative modeling to model time series. In this work we first establish necessary conditions for TFM to learn time series data. Next, we present a reparameterization trick which improves training stability. Finally, we adapt TFM to the clinical time series setting, demonstrating improved performance on three clinical time series datasets both in terms of absolute performance and uncertainty prediction.
2024-10-28
ArXiv (preprint)
doi.org
arxiv.org
In-Simulation Testing of Deep Learning Vision Models in Autonomous Robotic Manipulators
Dmytro Humeniuk
Houssem Ben Braiek
Thomas Reid
Foutse Khomh
2024-10-27
Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering (published)
doi.org
arxiv.org
Brain-like learning with exponentiated gradients
Jonathan Cornford
Roman Pogodin
Arna Ghosh
Kaiwen Sheng
Brendan A. Bicknell
Olivier Codol
Beverley A. Clark
Guillaume Lajoie
Blake Richards
2024-10-26
bioRxiv (preprint)
doi.org
Gravitational-Wave Parameter Estimation in non-Gaussian noise using Score-Based Likelihood Characterization
Ronan Legin
Maximiliano Isi
Kaze W. K. Wong
Yashar Hezaveh
Laurence Perreault-Levasseur
2024-10-25
ArXiv (preprint)
arxiv.org
arxiv.org
Prediction of Final Phosphorus Content of Steel in a Scrap-Based Electric Arc Furnace Using Artificial Neural Networks
Riadh Azzaz
Valentin Hurel
Patrice Ménard
M. Jahazi
Samira Ebrahimi Kahou
Elmira Moosavi-Khoonsari
2024-10-25
ArXiv (preprint)
doi.org
arxiv.org
scMoE: single-cell mixture of experts for learning hierarchical, cell-type-specific, and interpretable representations from heterogeneous scRNA-seq data
Michael Huang
Yue Li
2024-10-25
bioRxiv (preprint)
doi.org
Sparse Decomposition of Graph Neural Networks
Yaochen Hu
Mai Zeng
Ge Zhang
Pavel Rumiantsev
Liheng Ma
Yingxue Zhang
Mark Coates
2024-10-25
ArXiv (preprint)
doi.org
arxiv.org
Understanding Adam Requires Better Rotation Dependent Assumptions
Lucas Maes
Tianyue H. Zhang
Alexia Jolicoeur-Martineau
Ioannis Mitliagkas
Damien Scieur
Simon Lacoste-Julien
Charles Guille-Escuret
Despite its widespread adoption, Adam's advantage over Stochastic Gradient Descent (SGD) lacks a comprehensive theoretical explanation. This… (see more) paper investigates Adam's sensitivity to rotations of the parameter space. We demonstrate that Adam's performance in training transformers degrades under random rotations of the parameter space, indicating a crucial sensitivity to the choice of basis. This reveals that conventional rotation-invariant assumptions are insufficient to capture Adam's advantages theoretically. To better understand the rotation-dependent properties that benefit Adam, we also identify structured rotations that preserve or even enhance its empirical performance. We then examine the rotation-dependent assumptions in the literature, evaluating their adequacy in explaining Adam's behavior across various rotation types. This work highlights the need for new, rotation-dependent theoretical frameworks to fully understand Adam's empirical success in modern machine learning tasks.
2024-10-25
ArXiv (preprint)
doi.org
arxiv.org
Considerations and recommendations from the ISMRM Diffusion Study Group for preclinical diffusion MRI: Part 3 -- Ex vivo imaging: data processing, comparisons with microscopy, and tractography
Kurt G Schilling
Amy F. D. Howard
Francesco Grussu
Andrada Ianus
Brian Hansen
Rachel L. C. Barrett
Manisha Aggarwal
Stijn Michielse
Fatima Nasrallah
W. Syeda
Nian Wang
Jelle Veraart
Alard J. Roebroeck
Andrew F Bagdasarian
Cornelius Eichner
Farshid Sepehrband
Jan Zimmermann
L. Soustelle
Christien Bowman
Benjamin C. Tendler … (see 38 more)
A. Hertanu
Ben Jeurissen
Marleen Verhoye
Lucio Frydman
Y. Looij
David C. Hike
Jeff F. Dunn
Karla L. Miller
Bennett A. Landman
N. Shemesh
Adam Anderson
Emilie McKinnon
Shawna Farquharson
Flavio Dell’ Acqua
Carlo M. Pierpaoli
Ivana Drobnjak
Alexander Leemans
K. Harkins
Maxime Descoteaux
Duan Xu
Hao Huang
Mathieu D. Santin
Samuel C. Grant
Andre Obenaus
Gene S Kim
Dan Wu
D. Bihan
S. Blackband
Luisa Ciobanu
E. Fieremans
Ruiliang Bai
T. Leergaard
Jiangyang Zhang
T. Dyrby
G. A. Johnson
Julien Cohen-Adad
Matthew D. Budde
Ileana Ozana Jelescu
2024-10-24
ArXiv (preprint)
arxiv.org
arxiv.org
ConvNTC: Convolutional neural tensor completion for predicting the disease-related miRNA pairs and cell-related drug pairs
Pei Liu
Xiao Liang
Yue Li
Jiawei Luo
2024-10-24
bioRxiv (preprint)
doi.org
Multi-agent cooperation through learning-aware policy gradients
Alexander Meulemans
Seijin Kobayashi
Johannes von Oswald
Nino Scherrer
Eric Elmoznino
Blake Richards
Guillaume Lajoie
Blaise Agüera y Arcas
João Sacramento
Self-interested individuals often fail to cooperate, posing a fundamental challenge for multi-agent learning. How can we achieve cooperation… (see more) among self-interested, independent learning agents? Promising recent work has shown that in certain tasks cooperation can be established between learning-aware agents who model the learning dynamics of each other. Here, we present the first unbiased, higher-derivative-free policy gradient algorithm for learning-aware reinforcement learning, which takes into account that other agents are themselves learning through trial and error based on multiple noisy trials. We then leverage efficient sequence models to condition behavior on long observation histories that contain traces of the learning dynamics of other agents. Training long-context policies with our algorithm leads to cooperative behavior and high returns on standard social dilemmas, including a challenging environment where temporally-extended action coordination is required. Finally, we derive from the iterated prisoner's dilemma a novel explanation for how and when cooperation arises among self-interested learning-aware agents.
2024-10-24
ArXiv (preprint)
doi.org
arxiv.org