Publications

MiRGraph: A hybrid deep learning approach to identify microRNA-target interactions by integrating heterogeneous regulatory network and genomic sequences
Pei Liu
Yang Liu
Jiawei Luo
Yuemei Li
2024-10-01
bioRxiv (preprint)
doi.org
VinePPO: Refining Credit Assignment in RL Training of LLMs
Amirhossein Kazemnejad
Milad Aghajohari
Eva Portelance
Alessandro Sordoni
Siva Reddy
Aaron Courville
Nicolas Le Roux
Large language models (LLMs) are increasingly applied to complex reasoning tasks that require executing several complex steps before receivi… (see more)ng any reward. Properly assigning credit to these steps is essential for enhancing model performance. Proximal Policy Optimization (PPO), a common reinforcement learning (RL) algorithm used for LLM finetuning, employs value networks to tackle credit assignment. However, recent approaches achieve strong results without it, raising questions about the efficacy of value networks in practice. In this work, we systematically evaluate the efficacy of value networks and reveal their significant shortcomings in reasoning-heavy LLM tasks, showing that they often produce poor estimate of expected return and barely outperform a random baseline when comparing alternative steps. This motivates our key question: Can improved credit assignment enhance RL training for LLMs? To address this, we propose VinePPO, a straightforward approach that leverages the flexibility of language environments to compute unbiased Monte Carlo-based estimates. Our method consistently outperforms PPO and other baselines across MATH and GSM8K datasets in less wall-clock time (up to 3.0x). Crucially, it achieves higher test accuracy for a given training accuracy, capturing more generalization signal per sample. These results emphasize the importance of accurate credit assignment in RL training of LLM.
2024-10-01
arXiv (preprint)
doi.org
proceedings.mlr.press
Were RNNs All We Needed?
Leo Feng
Frederick Tung
Mohamed Osama Ahmed
Yoshua Bengio
Hossein Hajimirsadeghi
2024-10-01
ArXiv (preprint)
doi.org
arxiv.org
Were RNNs All We Needed?
Leo Feng
Frederick Tung
Mohamed Osama Ahmed
Yoshua Bengio
Hossein Hajimirsadeghi
The introduction of Transformers in 2017 reshaped the landscape of deep learning. Originally proposed for sequence modelling, Transformers h… (see more)ave since achieved widespread success across various domains. However, the scalability limitations of Transformers - particularly with respect to sequence length - have sparked renewed interest in novel recurrent models that are parallelizable during training, offer comparable performance, and scale more effectively. In this work, we revisit sequence modelling from a historical perspective, focusing on Recurrent Neural Networks (RNNs), which dominated the field for two decades before the rise of Transformers. Specifically, we examine LSTMs (1997) and GRUs (2014). We demonstrate that by simplifying these models, we can derive minimal versions (minLSTMs and minGRUs) that (1) use fewer parameters than their traditional counterparts, (2) are fully parallelizable during training, and (3) achieve surprisingly competitive performance on a range of tasks, rivalling recent models including Transformers.
2024-10-01
ArXiv (preprint)
doi.org
arxiv.org
Challenges for impact evaluation of WHO’s normative output
Catherine Régis
Gaëlle Foucault
Jean-Louis Denis
Pierre Larouche
Miriam Cohen
2024-09-30
Bulletin of the World Health Organization (published)
doi.org
Efficient line search for optimizing Area Under the ROC Curve in gradient descent
Jadon Fowler
Toby Dylan Hocking
Receiver Operating Characteristic (ROC) curves are useful for evaluation in binary classification and changepoint detection, but difficult t… (see more)o use for learning since the Area Under the Curve (AUC) is piecewise constant (gradient zero almost everywhere). Recently the Area Under Min (AUM) of false positive and false negative rates has been proposed as a differentiable surrogate for AUC. In this paper we study the piecewise linear/constant nature of the AUM/AUC, and propose new efficient path-following algorithms for choosing the learning rate which is optimal for each step of gradient descent (line search), when optimizing a linear model. Remarkably, our proposed line search algorithm has the same log-linear asymptotic time complexity as gradient descent with constant step size, but it computes a complete representation of the AUM/AUC as a function of step size. In our empirical study of binary classification problems, we verify that our proposed algorithm is fast and exact; in changepoint detection problems we show that the proposed algorithm is just as accurate as grid search, but faster.
2024-09-30
arXiv (published)
doi.org
arxiv.org
Finite Sample Complexity Analysis of Binary Segmentation
Toby Dylan Hocking
Binary segmentation is the classic greedy algorithm which recursively splits a sequential data set by optimizing some loss or likelihood fun… (see more)ction. Binary segmentation is widely used for changepoint detection in data sets measured over space or time, and as a sub-routine for decision tree learning. In theory it should be extremely fast for
2024-09-30
arXiv (published)
doi.org
arxiv.org
Inferring electric vehicle charging patterns from smart meter data for impact studies
Feng Li
Élodie Campeau
Ilhan Kocar
Antoine Lesage-Landry
2024-09-30
Electric power systems research (published)
doi.org
Innovative transfusion strategies for blood deserts in disaster settings
Shreenik Kundu
Ayla Gerk
Robert Glatter
Dan Poenaru
2024-09-30
BMJ Global Health (published)
doi.org
Long-term outcomes of critically ill patients with hematological malignancies: what is the impact of the coronavirus disease 2019 pandemic? Author's reply
Laveena Munshi
Guillaume Dumas
Sangeeta Mehta
2024-09-30
Intensive Care Medicine (published)
doi.org
MAP: Model Merging with Amortized Pareto Front Using Limited Computation
Li Li
Tianyu Zhang
Zhiqi Bu
Suyuchen Wang
Huan He
Jie Fu
Yonghui Wu
Jiang Bian
Yong Chen
Yoshua Bengio
2024-09-30
NeurIPS.cc/2024/Workshop/Federated_Learning (oral)
openreview.net
openreview.net
ProtSCAPE: Mapping the landscape of protein conformations in molecular dynamics
Siddharth Viswanath
Dhananjay Bhaskar
David R. Johnson
João Felipe Rocha
Egbert Castro
Jackson Grady
Alex T. Grigas
Michael Perlmutter
Corey S. O'Hern
Smita Krishnaswamy
Understanding the dynamic nature of protein structures is essential for comprehending their biological functions. While significant progress… (see more) has been made in predicting static folded structures, modeling protein motions on microsecond to millisecond scales remains challenging. To address these challenges, we introduce a novel deep learning architecture, Protein Transformer with Scattering, Attention, and Positional Embedding (ProtSCAPE), which leverages the geometric scattering transform alongside transformer-based attention mechanisms to capture protein dynamics from molecular dynamics (MD) simulations. ProtSCAPE utilizes the multi-scale nature of the geometric scattering transform to extract features from protein structures conceptualized as graphs and integrates these features with dual attention structures that focus on residues and amino acid signals, generating latent representations of protein trajectories. Furthermore, ProtSCAPE incorporates a regression head to enforce temporally coherent latent representations.
2024-09-30
arXiv (published)
doi.org
arxiv.org