Publications

Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Models
Lucas Berry
Axel Brando
David Meger
Generative diffusion models, notable for their large parameter count (exceeding 100 million) and operation within high-dimensional image spa… (see more)ces, pose significant challenges for traditional uncertainty estimation methods due to computational demands. In this work, we introduce an innovative framework, Diffusion Ensembles for Capturing Uncertainty (DECU), designed for estimating epistemic uncertainty for diffusion models. The DECU framework introduces a novel method that efficiently trains ensembles of conditional diffusion models by incorporating a static set of pre-trained parameters, drastically reducing the computational burden and the number of parameters that require training. Additionally, DECU employs Pairwise-Distance Estimators (PaiDEs) to accurately measure epistemic uncertainty by evaluating the mutual information between model outputs and weights in high-dimensional spaces. The effectiveness of this framework is demonstrated through experiments on the ImageNet dataset, highlighting its capability to capture epistemic uncertainty, specifically in under-sampled image classes.
2024-09-12
Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence (published)
doi.org
openreview.net
Are Heterophily-Specific GNNs and Homophily Metrics Really Effective? Evaluation Pitfalls and New Benchmarks
Sitao Luan
Qincheng Lu
Chenqing Hua
Xinyu Wang
Jiaqi Zhu
Xiao-Wen Chang
Guy Wolf
Jian Tang
Over the past decade, Graph Neural Networks (GNNs) have achieved great success on machine learning tasks with relational data. However, rece… (see more)nt studies have found that heterophily can cause significant performance degradation of GNNs, especially on node-level tasks. Numerous heterophilic benchmark datasets have been put forward to validate the efficacy of heterophily-specific GNNs and various homophily metrics have been designed to help people recognize these malignant datasets. Nevertheless, there still exist multiple pitfalls that severely hinder the proper evaluation of new models and metrics. In this paper, we point out three most serious pitfalls: 1) a lack of hyperparameter tuning; 2) insufficient model evaluation on the real challenging heterophilic datasets; 3) missing quantitative evaluation benchmark for homophily metrics on synthetic graphs. To overcome these challenges, we first train and fine-tune baseline models on
2024-09-09
ArXiv (preprint)
doi.org
arxiv.org
Correction: Economic evaluation of the effect of needle and syringe programs on skin, soft tissue, and vascular infections in people who inject drugs: a microsimulation modelling approach
Jihoon Lim
W Alton Russell
Mariam El-Sheikh
David Buckeridge
Dimitra Panagiotoglou
2024-09-09
Harm Reduction Journal (published)
doi.org
Perspectives on virtual interviews and emerging technologies integration in family medicine residency programs: a cross-sectional survey study
Raymond Tolentino
Charo Rodriguez
Fanny Hersson-Edery
Julie Lane
Samira Abbasgholizadeh-Rahimi
2024-09-09
BMC Medical Education (published)
doi.org
Accelerating Training with Neuron Interaction and Nowcasting Networks
Boris Knyazev
Abhinav Moudgil
Guillaume Lajoie
Eugene Belilovsky
Simon Lacoste-Julien
Neural network training can be accelerated when a learnable update rule is used in lieu of classic adaptive optimizers (e.g. Adam). However,… (see more) learnable update rules can be costly and unstable to train and use. A simpler recently proposed approach to accelerate training is to use Adam for most of the optimization steps and periodically, only every few steps, nowcast (predict future) parameters. We improve this approach by Neuron interaction and Nowcasting (NiNo) networks. NiNo leverages neuron connectivity and graph neural networks to more accurately nowcast parameters by learning in a supervised way from a set of training trajectories over multiple tasks. We show that in some networks, such as Transformers, neuron connectivity is non-trivial. By accurately modeling neuron connectivity, we allow NiNo to accelerate Adam training by up to 50\% in vision and language tasks.
2024-09-06
ArXiv (preprint)
doi.org
arxiv.org
The Strength of Fuel Refueling Location Problem Formulations
Nagisa Sugishita
Margarida Carvalho
Ribal Atallah
2024-09-06
ArXiv (preprint)
arxiv.org
arxiv.org
CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
Luke Rowe
Roger Girgis
Anthony Gosselin
Bruno Carrez
Florian Golemo
Felix Heide
Liam Paull
Chris Pal
Evaluating autonomous vehicle stacks (AVs) in simulation typically involves replaying driving logs from real-world recorded traffic. However… (see more), agents replayed from offline data do not react to the actions of the AV, and their behaviour cannot be easily controlled to simulate counterfactual scenarios. Existing approaches have attempted to address these shortcomings by proposing methods that rely on heuristics or learned generative models of real-world data but these approaches either lack realism or necessitate costly iterative sampling procedures to control the generated behaviours. In this work, we take an alternative approach and propose CtRL-Sim, a method that leverages return-conditioned offline reinforcement learning within a physics-enhanced Nocturne simulator to efficiently generate reactive and controllable traffic agents. Specifically, we process real-world driving data through the Nocturne simulator to generate a diverse offline reinforcement learning dataset, annotated with various reward terms. With this dataset, we train a return-conditioned multi-agent behaviour model that allows for fine-grained manipulation of agent behaviours by modifying the desired returns for the various reward components. This capability enables the generation of a wide range of driving behaviours beyond the scope of the initial dataset, including those representing adversarial behaviours. We demonstrate that CtRL-Sim can efficiently generate diverse and realistic safety-critical scenarios while providing fine-grained control over agent behaviours. Further, we show that fine-tuning our model on simulated safety-critical scenarios generated by our model enhances this controllability.
2024-09-05
robot-learning.org/CoRL/2024/Conference (accepted)
doi.org
openreview.net
Towards Robust Saliency Maps
Nham Le
Arie Gurfinkel
Xujie Si
Chuqin Geng
Saliency maps are one of the most popular tools to interpret the operation of a neural network: they compute input features deemed relevant … (see more)to the final prediction, which are often subsets of pixels that are easily understandable by a human being. However, it is known that relying solely on human assessment to judge a saliency map method can be misleading. In this work, we propose a new neural network verification specification called saliency-robustness, which aims to use formal methods to prove a relationship between Vanilla Gradient (VG) -- a simple yet surprisingly effective saliency map method -- and the network's prediction: given a network, if an input
2024-09-05
ACML.org/2024/Conference (published)
openreview.net
openreview.net
Nteasee: A mixed methods study of expert and general population perspectives on deploying AI for health in African countries
Mercy Nyamewaa Asiedu
Iskandar Haykel
Awa Dieng
K. Kauer
Tousif Ahmed
Florence Ofori
Charisma Chan
Stephen R. Pfohl
Negar Rostamzadeh
Katherine Heller
2024-09-04
ArXiv (preprint)
doi.org
arxiv.org
Reputation Gaming in Crowd Technical Knowledge Sharing
Iren Mazloomzadeh
Gias Uddin
Foutse Khomh
Ashkan Sami
Stack Overflow incentive system awards users with reputation scores to ensure quality. The decentralized nature of the forum may make the in… (see more)centive system prone to manipulation. This paper offers, for the first time, a comprehensive study of the reported types of reputation manipulation scenarios that might be exercised in Stack Overflow and the prevalence of such reputation gamers by a qualitative study of 1,697 posts from meta Stack Exchange sites. We found four different types of reputation fraud scenarios, such as voting rings where communities form to upvote each other repeatedly on similar posts. We developed algorithms that enable platform managers to automatically identify these suspicious reputation gaming scenarios for review. The first algorithm identifies isolated/semi-isolated communities where probable reputation frauds may occur mostly by collaborating with each other. The second algorithm looks for sudden unusual big jumps in the reputation scores of users. We evaluated the performance of our algorithms by examining the reputation history dashboard of Stack Overflow users from the Stack Overflow website. We observed that around 60-80% of users flagged as suspicious by our algorithms experienced reductions in their reputation scores by Stack Overflow.
2024-09-04
ACM Transactions on Software Engineering and Methodology (published)
doi.org
Advancing EDGE Zones to identify spatial conservation priorities of tetrapod evolutionary history
Sebastian Pipins
Jonathan E. M. Baillie
Alex Bowmer
Laura J. Pollock
Nisha Owen
Rikki Gumbs
2024-09-03
Nature Communications (published)
doi.org
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Usman Anwar
Abulhair Saparov
Javier Rando
Daniel Paleka
Miles Turpin
Peter Hase
Ekdeep Singh Lubana
Erik Jenner
Stephen Casper
Oliver Sourbut
Benjamin L. Edelman
Zhaowei Zhang
Mario Günther
Anton Korinek
Jose Hernandez-Orallo
Lewis Hammond
Eric J Bigelow
Alexander Pan
Lauro Langosco
Tomasz Korbak … (see 22 more)
Heidi Chenyu Zhang
Ruiqi Zhong
Sean O hEigeartaigh
Gabriel Recchia
Giulio Corsi
Alan Chan
Markus Anderljung
Lilian Edwards
Aleksandar Petrov
Yoshua Bengio
Christian Schroeder de Witt
Danqi Chen
Sumeet Ramesh Motwani
Samuel Albanie
Tegan Maharaj
Jakob Nicolaus Foerster
Philip Torr
Florian Tramèr
He He
Atoosa Kasirzadeh
Yejin Choi
David Scott Krueger
2024-09-02
TMLR (accepted)
doi.org
openreview.net