Publications

Survey of Scientific Rigor Studied in Machine Learning
D. Sculley
Gary R. Holt
Daniel R. Golovin
Eugene V. Davydov
Todd Phillips
Dietmar Ebner
Michael Young
Jean-francois Crespo
Dan Dennison
Emily Fox
Hugo Larochelle
The concern that Artificial Intelligence (AI) and Machine Learning (ML) are entering a “reproducibility crisis” has spurred significant … (voir plus)research in the past few years. Yet with each paper, it is often unclear what someone means by “reproducibility” and where it fits in the larger scope of what we will call the “scientific rigor” literature. Ultimately, the lack of clear rigor standards can affect the manner in which businesses seeking to adopt AI/ML implement such capabilities. In this survey, we will use 66 papers published since 2017 to construct a proposed set of 8 high-level categories of scientific rigor, what they are, and the history of work conducted in each. Our proposal is that these eight rigor types are not mutually exclusive and present a model for how they influence each other. To encourage more to study these questions, we map these rigors to the adoption process in real-world business use cases. In doing so, we can quantify gaps in the literature that suggest an under focus on the issues necessary for scientific rigor research to transition to practice
2023-01-01
(publié)
www.semanticscholar.org
Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning
Sébastien Lachapelle
Tristan Deleu
Divyat Mahajan
Ioannis Mitliagkas
Yoshua Bengio
Simon Lacoste-Julien
Quentin Bertrand
Although disentangled representations are often said to be beneficial for downstream tasks, current empirical and theoretical understanding … (voir plus)is limited. In this work, we provide evidence that disentangled representations coupled with sparse base-predictors improve generalization. In the context of multi-task learning, we prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem. Finally, we explore a meta-learning version of this algorithm based on group Lasso multiclass SVM base-predictors, for which we derive a tractable dual formulation. It obtains competitive results on standard few-shot classification benchmarks, while each task is using only a fraction of the learned representations.
2023-01-01
ICML (publié)
openreview.net
openreview.net
Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning
Sébastien Lachapelle
Tristan Deleu
Divyat Mahajan
Ioannis Mitliagkas
Yoshua Bengio
Simon Lacoste-Julien
Quentin Bertrand
Although disentangled representations are often said to be beneficial for downstream tasks, current empirical and theoretical understanding … (voir plus)is limited. In this work, we provide evidence that disentangled representations coupled with sparse task-specific predictors improve generalization. In the context of multi-task learning, we prove a new identifiability result that provides conditions under which maximally sparse predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem. Finally, we explore a meta-learning version of this algorithm based on group Lasso multiclass SVM predictors, for which we derive a tractable dual formulation. It obtains competitive results on standard few-shot classification benchmarks, while each task is using only a fraction of the learned representations.
2023-01-01
ICML (publié)
proceedings.mlr.press
openreview.net
Syntactic Substitutability as Unsupervised Dependency Syntax
Jasper Jian
Siva Reddy
2023-01-01
EMNLP (publié)
doi.org
openreview.net
Technology-enhanced trauma training in low-resource settings: A scoping review and feasibility analysis of educational technologies.
Minahil Khan
Fabio Botelho
Laura Pinkham
Elena Guadagno
Dan Poenaru
2023-01-01
Journal of Pediatric Surgery (publié)
doi.org
Temporal Abstraction in Reinforcement Learning with the Successor Representation
Marlos C. Machado
Andre Barreto
Doina Precup
Michael Bowling
arxiv.org
Test-time Defense against Adversarial Attacks: Detection and Reconstruction of Adversarial Examples via Masked Autoencoder
Yun-Yun Tsai
Ju-Chin Chao
Albert Wen
Zhaoyuan Yang
Chengzhi Mao
Tapan Shah
Junfeng Yang
Existing defense methods against adversarial attacks can be categorized into training time and test time defenses. Training time defense, i.… (voir plus)e., adversarial training, requires a significant amount of extra time for training and is often not able to be generalized to unseen attacks. On the other hand, test time defense by test time weight adaptation requires access to perform gradient descent on (part of) the model weights, which could be infeasible for models with frozen weights. To address these challenges, we propose DRAM, a novel defense method to Detect and Reconstruct the multiple types of Adversarial attacks via Masked autoencoder (MAE). We demonstrate how to use MAE losses to build a KS-test to detect adversarial attacks. Moreover, the MAE losses can be used to repair adversarial samples from unseen attack types. In this sense, DRAM neither requires model weight updates in test time nor augments the training set with more adversarial samples. Evaluating DRAM on the large-scale ImageNet data, we achieve the best detection rate of 82% on average on eight types of adversarial attacks compared with other detection baselines. For reconstruction, DRAM improves the robust accuracy by 6% ∼ 41% for Standard ResNet50 and 3% ∼ 8% for Robust ResNet50 compared with other self-supervision tasks, such as rotation prediction and contrastive learning.
2023-01-01
(publié)
www.semanticscholar.org
The Age of Ransomware: A Survey on the Evolution, Taxonomy, and Research Directions
Salwa Razaulla
Claude Fachkha
Christine Markarian
Amjad Gawanmeh
Wathiq Mansoor
Benjamin Fung
Chadi Assi
The proliferation of ransomware has become a significant threat to cybersecurity in recent years, causing significant financial, reputationa… (voir plus)l, and operational damage to individuals and organizations. This paper aims to provide a comprehensive overview of the evolution of ransomware, its taxonomy, and its state-of-the-art research contributions. We begin by tracing the origins of ransomware and its evolution over time, highlighting the key milestones and major trends. Next, we propose a taxonomy of ransomware that categorizes different types of ransomware based on their characteristics and behavior. Subsequently, we review the existing research over several years in regard to detection, prevention, mitigation, and prediction techniques. Our extensive analysis, based on more than 150 references, has revealed that significant research, specifically 72.8%, has focused on detecting ransomware. However, a lack of emphasis has been placed on predicting ransomware. Additionally, of the studies focused on ransomware detection, a significant portion, 70%, have utilized Machine Learning methods. This study uncovers a range of shortcomings in research pertaining to real-time protection and identifying zero-day ransomware, and two issues specific to Machine Learning models. Adversarial machine learning exploitation and concept drift have been identified as under-researched areas in the field. This survey is a constructive roadmap for researchers interested in ransomware research matters.
2023-01-01
IEEE Access (publié)
doi.org
The Dormant Neuron Phenomenon in Deep Reinforcement Learning
Ghada Sokar
Rishabh Agarwal
Pablo Samuel Castro
Utku Evci
In this work we identify the dormant neuron phenomenon in deep reinforcement learning, where an agent's network suffers from an increasing n… (voir plus)umber of inactive neurons, thereby affecting network expressivity. We demonstrate the presence of this phenomenon across a variety of algorithms and environments, and highlight its effect on learning. To address this issue, we propose a simple and effective method (ReDo) that Recycles Dormant neurons throughout training. Our experiments demonstrate that ReDo maintains the expressive power of networks by reducing the number of dormant neurons and results in improved performance.
2023-01-01
ICML (publié)
doi.org
openreview.net
On the (Im)Possibility of Estimating Various Notions of Differential Privacy (short paper)
Daniele Gorla
Louis Jalouzot
Federica Granese
Catuscia Palamidessi
Pablo Piantanida
We analyze to what extent final users can infer information about the level of protection of their data when the data obfuscation mechanism … (voir plus)is a priori unknown to them (the so-called “black-box" scenario). In particular, we delve into the investigation of two notions of local differential privacy (LDP), namely 𝜀 -LDP and Rényi LDP. On one hand, we prove that, without any assumption on the underlying distributions, it is not possible to have an algorithm able to infer the level of data protection with provable guarantees. On the other hand, we demonstrate that, under reasonable assumptions (namely, Lipschitzness of the involved densities on a closed interval), such guarantees exist and can be achieved by a simple histogram-based estimator.
2023-01-01
ICTCS (publié)
dblp.uni-trier.de
On the Limitations of Elo: Real-World Games, are Transitive, not Additive
Quentin Bertrand
Wojciech M. Czarnecki
Gauthier Gidel
Real-world competitive games, such as chess, go, or StarCraft II, rely on Elo models to measure the strength of their players. Since these g… (voir plus)ames are not fully transitive, using Elo implicitly assumes they have a strong transitive component that can correctly be identified and extracted. In this study, we investigate the challenge of identifying the strength of the transitive component in games. First, we show that Elo models can fail to extract this transitive component, even in elementary transitive games. Then, based on this observation, we propose an extension of the Elo score: we end up with a disc ranking system that assigns each player two scores, which we refer to as skill and consistency. Finally, we propose an empirical validation on payoff matrices coming from real-world games played by bots and humans.
2023-01-01
AISTATS (publié)
doi.org
arxiv.org
The race to understand immunopathology in COVID-19: Perspectives on the impact of quantitative approaches to understand within-host interactions
Sonia Gazeau
Xiaoyan Deng
Hsu Kiang Ooi
Fatima Mostefai
Julie Hussin
Jane Heffernan
Adrianne L. Jenner
Morgan Craig
2023-01-01
ImmunoInformatics (publié)
doi.org