Publications

PAC-Bayesian Generalization Bounds for Adversarial Generative Models

Sokhna Diarra Mbacke

Florence Clerc

Pascal Germain

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

doi.org

openreview.net

Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design

Chuan Guo

Kamalika Chaudhuri

Pierre Stock

Michael Rabbat

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

proceedings.mlr.press

openreview.net

ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts

Minghao Xu

Xinyu Yuan

Santiago Miret

Jian Tang

Current protein language models (PLMs) learn protein representations mainly based on their sequences, thereby well capturing co-evolutionary… (voir plus) information, but they are unable to explicitly acquire protein functions, which is the end goal of protein representation learning. Fortunately, for many proteins, their textual property descriptions are available, where their various functions are also described. Motivated by this fact, we first build the ProtDescribe dataset to augment protein sequences with text descriptions of their functions and other important properties. Based on this dataset, we propose the ProtST framework to enhance Protein Sequence pre-training and understanding by biomedical Texts. During pre-training, we design three types of tasks, i.e., unimodal mask prediction, multimodal representation alignment and multimodal mask prediction, to enhance a PLM with protein property information with different granularities and, at the same time, preserve the PLM’s original representation power. On downstream tasks, ProtST enables both supervised learning and zero-shot prediction. We verify the superiority of ProtST-induced PLMs over previous ones on diverse representation learning benchmarks. Under the zero-shot setting, we show the effectiveness of ProtST on zero-shot protein classification, and ProtST also enables functional protein retrieval from a large-scale database without any function annotation.

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

doi.org

openreview.net

Regions of Reliability in the Evaluation of Multivariate Probabilistic Forecasts

Étienne Marcotte

Valentina Zantedeschi

Alexandre Drouin

Nicolas Chapados

Multivariate probabilistic time series forecasts are commonly evaluated via proper scoring rules, i.e., functions that are minimal in expect… (voir plus)ation for the ground-truth distribution. However, this property is not sufficient to guarantee good discrimination in the non-asymptotic regime. In this paper, we provide the first systematic finite-sample study of proper scoring rules for time-series forecasting evaluation. Through a power analysis, we identify the"region of reliability"of a scoring rule, i.e., the set of practical conditions where it can be relied on to identify forecasting errors. We carry out our analysis on a comprehensive synthetic benchmark, specifically designed to test several key discrepancies between ground-truth and forecast distributions, and we gauge the generalizability of our findings to real-world tasks with an application to an electricity production problem. Our results reveal critical shortcomings in the evaluation of multivariate probabilistic forecasts as commonly performed in the literature.

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

doi.org

openreview.net

Robust Perception through Equivariance

Chengzhi Mao

Lingyu Zhang

Abhishek Vaibhav Joshi

Junfeng Yang

Hao Wang

Carl Vondrick

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

doi.org

openreview.net

R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents

Daniel D. Johnson

Danny Tarlow

Christian Walder

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

doi.org

openreview.net

Sampling-Based Accuracy Testing of Posterior Estimators for General Inference

Pablo Lemos

Adam Coogan

Yashar Hezaveh

Laurence Perreault-Levasseur

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

doi.org

openreview.net

Uncertain Evidence in Probabilistic Models and Stochastic Simulators

Andreas Munk

Alexander Mead

Frank Wood

We consider the problem of performing Bayesian inference in probabilistic models where observations are accompanied by uncertainty, referred… (voir plus) to as "uncertain evidence.'' We explore how to interpret uncertain evidence, and by extension the importance of proper interpretation as it pertains to inference about latent variables. We consider a recently-proposed method "distributional evidence'' as well as revisit two older methods: Jeffrey's rule and virtual evidence. We devise guidelines on how to account for uncertain evidence and we provide new insights, particularly regarding consistency. To showcase the impact of different interpretations of the same uncertain evidence, we carry out experiments in which one interpretation is defined as "correct.'' We then compare inference results from each different interpretation illustrating the importance of careful consideration of uncertain evidence.

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

doi.org

openreview.net

Unlocking Slot Attention by Changing Optimal Transport Costs

Yan Zhang

David W Zhang

Simon Lacoste-Julien

Gertjan J. Burghouts

Cees G. M. Snoek

Slot attention is a powerful method for object-centric modeling in images and videos. However, its set-equivariance limits its ability to ha… (voir plus)ndle videos with a dynamic number of objects because it cannot break ties. To overcome this limitation, we first establish a connection between slot attention and optimal transport. Based on this new perspective we propose **MESH** (Minimize Entropy of Sinkhorn): a cross-attention module that combines the tiebreaking properties of unregularized optimal transport with the speed of regularized optimal transport. We evaluate slot attention using MESH on multiple object-centric learning benchmarks and find significant improvements over slot attention in every setting.

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

doi.org

openreview.net

Omega: Optimistic EMA Gradients

Juan Ramirez

Rohan Sukumaran

Quentin Bertrand

Gauthier Gidel

Stochastic min-max optimization has gained interest in the machine learning community with the advancements in GANs and adversarial training… (voir plus). Although game optimization is fairly well understood in the deterministic setting, some issues persist in the stochastic regime. Recent work has shown that stochastic gradient descent-ascent methods such as the optimistic gradient are highly sensitive to noise or can fail to converge. Although alternative strategies exist, they can be prohibitively expensive. We introduce Omega, a method with optimistic-like updates that mitigates the impact of noise by incorporating an EMA of historic gradients in its update rule. We also explore a variation of this algorithm that incorporates momentum. Although we do not provide convergence guarantees, our experiments on stochastic games show that Omega outperforms the optimistic gradient method when applied to linear players.

2023-07-02

ICML.cc/2023/Workshop/LXAI_Regular_Deadline (présentation orale)

doi.org

openreview.net

Aperiodic brain activity and response to anesthesia vary in disorders of consciousness

Charlotte Maschke

Catherine Duclos

Adrian M. Owen

Karim Jerbi

Stefanie Blain-Moraes

Stefanie

2023-07-01