Publications

Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets
Dominique Beaini
Shenyang Huang
Joao Alex Cunha
Zhiyi Li
Gabriela Moisescu-Pareja
Oleksandr Dymov
Samuel Maddrell-Mander
Callum McLean
Frederik Wenkel
Luis Müller
Jama Hussein Mohamud
Ali Parviz
Michael Craig
Michał Koziarski
Jiarui Lu
Zhaocheng Zhu
Cristian Gabellini
Kerstin Klaser
Josef Dean
Cas Wognum … (voir 14 de plus)
Maciej Sypetkowski
Guillaume Rabusseau
Reihaneh Rabbany
Jian Tang
Christopher Morris
Mirco Ravanelli
Guy Wolf
Prudencio Tossou
Hadrien Mary
Therence Bois
Andrew Fitzgibbon
Błażej Banaszewski
Chad Martin
Dominic Masters
Recently, pre-trained foundation models have shown significant advancements in multiple fields. However, the lack of datasets with labeled f… (voir plus)eatures and codebases has hindered the development of a supervised foundation model for molecular tasks. Here, we have carefully curated seven datasets specifically tailored for node- and graph-level prediction tasks to facilitate supervised learning on molecules. Moreover, to support the development of multi-task learning on our proposed datasets, we created the Graphium graph machine learning library. Our dataset collection encompasses two distinct categories. Firstly, the TOYMIX category modifies three small existing datasets with additional data for multi-task learning. Secondly, the LARGEMIX category includes four large-scale datasets with 344M graph-level data points and 409M node-level data points from ∼5M unique molecules. Finally, the ultra-large dataset contains 2,210M graph-level data points and 2,031M node-level data points coming from 86M molecules. Hence our datasets represent an order of magnitude increase in data volume compared to other 2D-GNN datasets. In addition, recognizing that molecule-related tasks often span multiple levels, we have designed our library to explicitly support multi-tasking, offering a diverse range of multi-level representations, i.e., representations at the graph, node, edge, and node-pair level. We equipped the library with an extensive collection of models and features to cover different levels of molecule analysis. By combining our curated datasets with this versatile library, we aim to accelerate the development of molecule foundation models. Datasets and code are available at https://github.com/datamol-io/graphium.
2024-05-06
International Conference on Learning Representations (Accept (poster))
doi.org
openreview.net
Tree Cross Attention
Leo Feng
Frederick Tung
Hossein Hajimirsadeghi
Yoshua Bengio
Mohamed Osama Ahmed
Cross Attention is a popular method for retrieving information from a set of context tokens for making predictions. At inference time, for e… (voir plus)ach prediction, Cross Attention scans the full set of
2024-05-06
International Conference on Learning Representations (Accept (poster))
doi.org
openreview.net
WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series
Jean-christophe Gagnon-audet
Kartik Ahuja
Mohammad-Javad Darvishi-Bayazi
Pooneh Mousavi
Guillaume Dumas
Irina Rish
2024-05-06
International Conference on Learning Representations (Accept (Poster))
doi.org
openreview.net
BETAC: Bidirectional Encoder Transformer for Assembly Code Function Name Recovery
Guillaume Breyton
Mohd Saqib
Benjamin C. M. Fung
Philippe Charland
Recovering function names from stripped binaries is a crucial and time-consuming task for software reverse engineering’ particularly in en… (voir plus)hancing network reliability, resilience, and security. This paper tackles the challenge of recovering function names in stripped binaries, a fundamental step in reverse engineering. The absence of syntactic information and the possibility of different code producing identical behavior complicate this task. To overcome these challenges, we introduce a novel model, the Bidirectional Encoder Transformer for Assembly Code (BETAC), leveraging a transformer-based architecture known for effectively processing sequential data. BETAC utilizes self-attention mechanisms and feed-forward networks to discern complex relationships within assembly code for precise function name prediction. We evaluated BETAC against various existing encoder and decoder models in diverse binary datasets, including benign and malicious codes in multiple formats. Our model demonstrated superior performance over previous techniques in certain metrics and showed resilience against code obfuscation.
2024-05-05
2024 20th International Conference on the Design of Reliable Communication Networks (DRCN) (publié)
doi.org
Linking aerial hyperspectral data to canopy tree biodiversity: An examination of the spectral variation hypothesis
Anna L. Crofts
Christine I. B. Wallis
Sabine St‐Jean
Sabrina Demers‐Thibeault
Deep Inamdar
J. Pablo Arroyo‐Mora
Margaret Kalácska
Étienne Laliberté
Mark Vellend
Imaging spectroscopy is emerging as a leading remote sensing method for quantifying plant biodiversity. The spectral variation hypothesis pr… (voir plus)edicts that variation in plant hyperspectral reflectance is related to variation in taxonomic and functional identity. While most studies report some correlation between spectral and field‐based (i.e., taxonomic and functional) expressions of biodiversity, the observed strength of association is highly variable, and the utility in applying spectral community properties to examine environmental drivers of communities remains unknown. We linked hyperspectral data acquired by airborne imaging spectrometers with precisely geolocated field plots to examine the spectral variation hypothesis along a temperate‐to‐boreal forest gradient in southern Québec, Canada. First, we examine the degree of association between spectral and field‐based dimensions of canopy tree composition and diversity. Second, we ask whether the relationships between field‐based community properties and the environment are reproduced when using spectral community properties. We found support for the spectral variation hypothesis with the strength of association generally greater for the functional than taxonomic dimension, but the strength of relationships was highly variable and dependent on the choice of method or metric used to quantify spectral and field‐based community properties. Using a multivariate approach (comparisons of separate ordinations), spectral composition was moderately well correlated with field‐based composition; however, the degree of association increased when univariately relating the main axes of compositional variation. Spectral diversity was most tightly associated with functional diversity metrics that quantify functional richness and divergence. For predicting canopy tree composition and diversity using environmental variables, the same qualitative conclusions emerge when hyperspectral or field‐based data are used. Spatial patterns of canopy tree community properties were strongly related to the turnover from temperate‐to‐boreal communities, with most variation explained by elevation. Spectral composition and diversity provide a straightforward way to quantify plant biodiversity across large spatial extents without the need for a priori field observations. While commonly framed as a potential tool for biodiversity monitoring, we show that spectral community properties can be applied more widely to assess the environmental drivers of biodiversity, thereby helping to advance our understanding of the drivers of biogeographical patterns of plant communities.
2024-05-05
Ecological Monographs (publié)
doi.org
Minimax Exploiter: A Data Efficient Approach for Competitive Self-Play
Daniel Bairamian
Philippe Marcotte
Joshua Romoff
Gabriel Robert
Derek Nowrouzezahrai
Recent advances in Competitive Self-Play (CSP) have achieved, or even surpassed, human level performance in complex game environments such a… (voir plus)s Dota 2 and StarCraft II using Distributed Multi-Agent Reinforcement Learning (MARL). One core component of these methods relies on creating a pool of learning agents -- consisting of the Main Agent, past versions of this agent, and Exploiter Agents -- where Exploiter Agents learn counter-strategies to the Main Agents. A key drawback of these approaches is the large computational cost and physical time that is required to train the system, making them impractical to deploy in highly iterative real-life settings such as video game productions. In this paper, we propose the Minimax Exploiter, a game theoretic approach to exploiting Main Agents that leverages knowledge of its opponents, leading to significant increases in data efficiency. We validate our approach in a diversity of settings, including simple turn based games, the arcade learning environment, and For Honor, a modern video game. The Minimax Exploiter consistently outperforms strong baselines, demonstrating improved stability and data efficiency, leading to a robust CSP-MARL method that is both flexible and easy to deploy.
2024-05-05
International Joint Conference on Autonomous Agents and Multiagent Systems (publié)
doi.org
arxiv.org
A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem
Paul Barde
Jakob Foerster
Derek Nowrouzezahrai
Amy Zhang
Training multiple agents to coordinate is an essential problem with applications in robotics, game theory, economics, and social sciences. H… (voir plus)owever, most existing Multi-Agent Reinforcement Learning (MARL) methods are online and thus impractical for real-world applications in which collecting new interactions is costly or dangerous. While these algorithms should leverage offline data when available, doing so gives rise to what we call the offline coordination problem. Specifically, we identify and formalize the strategy agreement (SA) and the strategy fine-tuning (SFT) coordination challenges, two issues at which current offline MARL algorithms fail. Concretely, we reveal that the prevalent model-free methods are severely deficient and cannot handle coordination-intensive offline multi-agent tasks in either toy or MuJoCo domains. To address this setback, we emphasize the importance of inter-agent interactions and propose the very first model-based offline MARL method. Our resulting algorithm, Model-based Offline Multi-Agent Proximal Policy Optimization (MOMA-PPO) generates synthetic interaction data and enables agents to converge on a strategy while fine-tuning their policies accordingly. This simple model-based solution solves the coordination-intensive offline tasks, significantly outperforming the prevalent model-free methods even under severe partial observability and with learned world models.
2024-05-05
International Joint Conference on Autonomous Agents and Multiagent Systems (publié)
doi.org
arxiv.org
Characterizing the voxel-based approaches in radioembolization dosimetry with reDoseMC.
Taehyung Peter Kim
S. Enger
BACKGROUND Yttrium-90 ( 90 Y …
2024-05-03
Medical Physics (publié)
doi.org
A Comprehensive Dataset of Four Provincial Legislative Assembly Members
Alex B. Rivard
Marc André Bodet
Jean-François Godbout
Éric Montigny
This research note reports on a new dataset about legislators in four Canadian provinces since the establishment of their colonial assemblie… (voir plus)s in the eighteenth century. Over 7,000 legislators from Ontario, Quebec, New Brunswick, and Nova Scotia are included, with consolidated information drawn from multiple sources about parliamentarians’ years of birth and death, religion, electoral performance, kinship, and several other biographical indicators. We also illustrate the utility of such data with the help of a few descriptive examples drawn from the four provinces. We believe this consolidated dataset offers several opportunities for future research on representation, legislative activities and party politics.
2024-05-02
Canadian Journal of Political Science/Revue canadienne de science politique (publié)
doi.org
Hierarchies define the scalability of robot swarms
Vivek Shankar Vardharajan
Karthik Soma
Sepand Dyanatkar
Pierre-Yves Lajoie
Giovanni Beltrame
The emerging behaviors of swarms have fascinated scientists and gathered significant interest in the field of robotics. Traditionally, swarm… (voir plus)s are viewed as egalitarian, with robots sharing identical roles and capabilities. However, recent findings highlight the importance of hierarchy for deploying robot swarms more effectively in diverse scenarios. Despite nature's preference for hierarchies, the robotics field has clung to the egalitarian model, partly due to a lack of empirical evidence for the conditions favoring hierarchies. Our research demonstrates that while egalitarian swarms excel in environments proportionate to their collective sensing abilities, they struggle in larger or more complex settings. Hierarchical swarms, conversely, extend their sensing reach efficiently, proving successful in larger, more unstructured environments with fewer resources. We validated these concepts through simulations and physical robot experiments, using a complex radiation cleanup task. This study paves the way for developing adaptable, hierarchical swarm systems applicable in areas like planetary exploration and autonomous vehicles. Moreover, these insights could deepen our understanding of hierarchical structures in biological organisms.
2024-05-02
ArXiv (prépublication)
doi.org
arxiv.org
Conditions on Preference Relations that Guarantee the Existence of Optimal Policies
Jonathan Colaço Carr
Prakash Panangaden
Doina Precup
Learning from Preferential Feedback (LfPF) plays an essential role in training Large Language Models, as well as certain types of interactiv… (voir plus)e learning agents. However, a substantial gap exists between the theory and application of LfPF algorithms. Current results guaranteeing the existence of optimal policies in LfPF problems assume that both the preferences and transition dynamics are determined by a Markov Decision Process. We introduce the Direct Preference Process, a new framework for analyzing LfPF problems in partially-observable, non-Markovian environments. Within this framework, we establish conditions that guarantee the existence of optimal policies by considering the ordinal structure of the preferences. We show that a decision-making problem can have optimal policies -- that are characterized by recursive optimality equations -- even when no reward function can express the learning goal. These findings underline the need to explore preference-based learning strategies which do not assume that preferences are generated by reward.
2024-05-01
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press
E(3)-Equivariant Mesh Neural Networks
Thuan Trang
Nhat Khang Ngo
Daniel Levy
Thieu N. Vo
Siamak Ravanbakhsh
Truong Son Hy
Triangular meshes are widely used to represent three-dimensional objects. As a result, many recent works have address the need for geometric… (voir plus) deep learning on 3D mesh. However, we observe that the complexities in many of these architectures does not translate to practical performance, and simple deep models for geometric graphs are competitive in practice. Motivated by this observation, we minimally extend the update equations of E(n)-Equivariant Graph Neural Networks (EGNNs) (Satorras et al., 2021) to incorporate mesh face information, and further improve it to account for long-range interactions through hierarchy. The resulting architecture, Equivariant Mesh Neural Network (EMNN), outperforms other, more complicated equivariant methods on mesh tasks, with a fast run-time and no expensive pre-processing. Our implementation is available at https://github.com/HySonLab/EquiMesh
2024-05-01
International Conference on Artificial Intelligence and Statistics (Accept (Poster))
doi.org
proceedings.mlr.press