Publications

An Analysis of Quantile Temporal-Difference Learning
Mark Rowland
Remi Munos
Mohammad Gheshlaghi Azar
Yunhao Tang
Georg Ostrovski
Anna Harutyunyan
K. Tuyls
Bellemare Marc-Emmanuel
Will Dabney
We analyse quantile temporal-difference learning (QTD), a distributional reinforcement learning algorithm that has proven to be a key compon… (voir plus)ent in several successful large-scale applications of reinforcement learning. Despite these empirical successes, a theoretical understanding of QTD has proven elusive until now. Unlike classical TD learning, which can be analysed with standard stochastic approximation tools, QTD updates do not approximate contraction mappings, are highly non-linear, and may have multiple fixed points. The core result of this paper is a proof of convergence to the fixed points of a related family of dynamic programming procedures with probability 1, putting QTD on firm theoretical footing. The proof establishes connections between QTD and non-linear differential inclusions through stochastic approximation theory and non-smooth analysis.
2023-12-31
Journal of Machine Learning Research (publié)
doi.org
arxiv.org
An Analytic Hierarchy Process based approach for assessing the performance of photovoltaic solar power plants
Meryam Chafiq
Loubna Benabbou
Hanane Dagdougui
Ismail Belhaj
Abdelali Djdiaa
Hicham Bouzekri
Abdelaziz Berrado
2023-12-31
IFAC-PapersOnLine (publié)
doi.org
Application-Driven Innovation in Machine Learning
David Rolnick
Alán Aspuru-Guzik
Sara Beery
Bistra Dilkina
Priya L. Donti
Marzyeh Ghassemi
Hannah Kerner
Claire Monteleoni
Esther Rolf
Milind Tambe
Adam White
As applications of machine learning proliferate, innovative algorithms inspired by specific real-world challenges have become increasingly i… (voir plus)mportant. Such work offers the potential for significant impact not merely in domains of application but also in machine learning itself. In this paper, we describe the paradigm of application-driven research in machine learning, contrasting it with the more standard paradigm of methods-driven research. We illustrate the benefits of application-driven machine learning and how this approach can productively synergize with methods-driven work. Despite these benefits, we find that reviewing, hiring, and teaching practices in machine learning often hold back application-driven innovation. We outline how these processes may be improved.
2023-12-31
ICML (publié)
doi.org
arxiv.org
Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared Task
Markus Freitag
Nitika Mathur
Daniel Deutsch
Chi-kiu Lo
Eleftherios Avramidis
Ricardo Rei
Brian Thompson
Frédéric Blain
Tom Kocmi
Jiayi Wang
David Ifeoluwa Adelani
Marianna Buchicchio
Chrysoula Zerva
2023-12-31
Conference on Machine Translation (publié)
doi.org
ARGV: 3D genome structure exploration using augmented reality
Chrisostomos Drogaris
Yanlin Zhang
Eric Zhang
Elena Nazarova
Roman Sarrazin-Gendron
Sélik Wilhelm-Landry
Yan Cyr
Jacek Majewski
Mathieu Blanchette
Jérôme Waldispühl
Over the past two decades, scientists have increasingly realized the importance of the three-dimensional (3D) genome organization in regulat… (voir plus)ing cellular activity. Hi-C and related experiments yield 2D contact matrices that can be used to infer 3D models of chromosome structure. Visualizing and analyzing genomes in 3D space remains challenging. Here, we present ARGV, an augmented reality 3D Genome Viewer. ARGV contains more than 350 pre-computed and annotated genome structures inferred from Hi-C and imaging data. It offers interactive and collaborative visualization of genomes in 3D space, using standard mobile phones or tablets. A user study comparing ARGV to existing tools demonstrates its benefits.
2023-12-31
BMC Bioinformatics (publié)
doi.org
AsmDocGen: Generating Functional Natural Language Descriptions for Assembly Code
Jesia Yuki
Mohammadhossein Amouei
Benjamin C. M. Fung
Philippe Charland
Andrew Walenstein
2023-12-31
International Conference on Software and Data Technologies (publié)
doi.org
Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy
Danqi Liao
Chen Liu
Benjamin W Christensen
Alexander Tong
Guillaume Huguet
Guy Wolf
Maximilian Nickel
Ian Adelstein
Smita Krishnaswamy
Entropy and mutual information in neural networks provide rich information on the learning process, but they have proven difficult to comput… (voir plus)e reliably in high dimensions. Indeed, in noisy and high-dimensional data, traditional estimates in ambient dimensions approach a fixed entropy and are prohibitively hard to compute. To address these issues, we leverage data geometry to access the underlying manifold and reliably compute these information-theoretic measures. Specifically, we define diffusion spectral entropy (DSE) in neural representations of a dataset as well as diffusion spectral mutual information (DSMI) between different variables representing data. First, we show that they form noise-resistant measures of intrinsic dimensionality and relationship strength in high-dimensional simulated data that outperform classic Shannon entropy, nonparametric estimation, and mutual information neural estimation (MINE). We then study the evolution of representations in classification networks with supervised learning, self-supervision, or overfitting. We observe that (1) DSE of neural representations increases during training; (2) DSMI with the class label increases during generalizable learning but stays stagnant during overfitting; (3) DSMI with the input signal shows differing trends: on MNIST it increases, while on CIFAR-10 and STL-10 it decreases. Finally, we show that DSE can be used to guide better network initialization and that DSMI can be used to predict downstream classification accuracy across 962 models on ImageNet.
2023-12-31
CISS (publié)
doi.org
openreview.net
Asymmetry in the complexity of the multi-commodity network pricing problem
Quang Minh Bui
Margarida Carvalho
José Neto
2023-12-31
Math. Program. (publié)
doi.org
arxiv.org
Attention-based Class-Conditioned Alignment for Multi-Source Domain Adaptation of Object Detectors
Atif Belal
Akhil Meethal
Francisco Perdigon Romero
Marco Pedersoli
Eric Granger
Domain adaptation methods for object detection (OD) strive to mitigate the impact of distribution shifts by promoting feature alignment acro… (voir plus)ss source and target domains. Multi-source domain adaptation (MSDA) allows leveraging multiple annotated source datasets and unlabeled target data to improve the accuracy and robustness of the detection model. Most state-of-the-art MSDA methods for OD perform feature alignment in a class-agnostic manner. This is challenging since the objects have unique modality information due to variations in object appearance across domains. A recent prototype-based approach proposed a class-wise alignment, yet it suffers from error accumulation caused by noisy pseudo-labels that can negatively affect adaptation with imbalanced data. To overcome these limitations, we propose an attention-based class-conditioned alignment method for MSDA, designed to align instances of each object category across domains. In particular, an attention module combined with an adversarial domain classifier allows learning domain-invariant and class-specific instance representations. Experimental results on multiple benchmarking MSDA datasets indicate that our method outperforms state-of-the-art methods and exhibits robustness to class imbalance, achieved through a conceptually simple class-conditioning strategy. Our code is available at: https://github.com/imatif17/ACIA.
2023-12-31
arXiv.org (prépublication)
doi.org
arxiv.org
An Attentive Approach for Building Partial Reasoning Agents from Pixels
Safa Alver
Doina Precup
We study the problem of building reasoning agents that are able to generalize in an effective manner. Towards this goal, we propose an end-t… (voir plus)o-end approach for building model-based reinforcement learning agents that dynamically focus their reasoning to the relevant aspects of the environment: after automatically identifying the distinct aspects of the environment, these agents dynamically filter out the relevant ones and then pass them to their simulator to perform partial reasoning. Unlike existing approaches, our approach works with pixel-based inputs and it allows for interpreting the focal points of the agent. Our quantitative analyses show that the proposed approach allows for effective generalization in high-dimensional domains with raw observational inputs. We also perform ablation analyses to validate our design choices. Finally, we demonstrate through qualitative analyses that our approach actually allows for building agents that focus their reasoning on the relevant aspects of the environment.
2023-12-31
Trans. Mach. Learn. Res. (publié)
openreview.net
openreview.net
Automatic Segmentation of the Spinal Cord Nerve Rootlets
Jan Valosek
Theo Mathieu
Raphaëlle Schlienger
Olivia S. Kowalczyk
Julien Cohen-Adad
Precise identification of spinal nerve rootlets is relevant to delineate spinal levels for the study of functional activity in the spinal co… (voir plus)rd. The goal of this study was to develop an automatic method for the semantic segmentation of spinal nerve rootlets from T2-weighted magnetic resonance imaging (MRI) scans. Images from two open-access MRI datasets were used to train a 3D multi-class convolutional neural network using an active learning approach to segment C2-C8 dorsal nerve rootlets. Each output class corresponds to a spinal level. The method was tested on 3T T2-weighted images from datasets unseen during training to assess inter-site, inter-session, and inter-resolution variability. The test Dice score was 0.67 +- 0.16 (mean +- standard deviation across testing images and rootlets levels), suggesting a good performance. The method also demonstrated low inter-vendor and inter-site variability (coefficient of variation <= 1.41 %), as well as low inter-session variability (coefficient of variation <= 1.30 %) indicating stable predictions across different MRI vendors, sites, and sessions. The proposed methodology is open-source and readily available in the Spinal Cord Toolbox (SCT) v6.2 and higher.
2023-12-31
Imaging Neuroscience (publié)
doi.org
arxiv.org
BAND: Biomedical Alert News Dataset
Zihao Fu
Meiru Zhang
Zaiqiao Meng
Yannan Shen
Anya Okhmatovskaia
David L Buckeridge
Nigel Collier
2023-12-31
AAAI (publié)
doi.org
arxiv.org