Publications

PMFL: Partial Meta-Federated Learning for heterogeneous tasks and its applications on real-world medical records
Tianyi Zhang
Shirui Zhang
Ziwei Chen
Yoshua Bengio
Dianbo Liu
Federated machine learning is a versatile and flexible tool to utilize distributed data from different sources, especially when communicatio… (see more)n technology develops rapidly and an unprecedented amount of data could be collected on mobile devices nowadays. Federated learning method exploits not only the data but the computational power of all devices in the network to achieve more efficient model training. Nevertheless, while most traditional federated learning methods work well for homogeneous data and tasks, adapting the method to heterogeneous data and task distribution is challenging. This limitation has constrained the applications of federated learning in real-world contexts, especially in healthcare settings. Inspired by the fundamental idea of meta-learning, in this study we propose a new algorithm, which is an integration of federated learning and meta-learning, to tackle this issue. In addition, owing to the advantage of transfer learning for model generalization, we further improve our algorithm by introducing partial parameter sharing to balance global and local learning. We name this method partial meta-federated learning (PMFL). Finally, we apply the algorithms to two medical datasets. We show that our algorithm could obtain the fastest training speed and achieve the best performance when dealing with heterogeneous medical datasets.
2021-12-31
IEEE Conference on Big Data (published)
doi.org
arxiv.org
PRACTICAL GUIDE
Alessio Mora
Irene Tenison
Paolo Bellavista
Irina Rish
2021-12-31
(published)
www.semanticscholar.org
Predicting Tactical Solutions to Operational Planning Problems under Imperfect Information
Eric Larsen
Sébastien Lachapelle
Yoshua Bengio
Emma Frejinger
Simon Lacoste-Julien
Andrea Lodi
2021-12-31
INFORMS Journal on Computing (published)
doi.org
arxiv.org
(Private)-Retroactive Carbon Pricing [(P)ReCaP]: A Market-based Approach for Climate Finance and Risk Assessment
Yoshua Bengio
Prateek Gupta
Dylan Radovic
Maarten Scholl
Andrew Williams
Christian Schroeder de Witt
Tianyu Zhang
Yang Zhang
Insufficient Social Cost of Carbon (SCC) estimation methods and short-term decision-making horizons have hindered the ability of carbon emit… (see more)ters to properly correct for the negative externalities of climate change, as well as the capacity of nations to balance economic and climate policy. To overcome these limitations, we introduce Retrospective Social Cost of Carbon Updating (ReSCCU), a novel mechanism that corrects for these limitations as empirically measured evidence is collected. To implement ReSCCU in the context of carbon taxation, we propose Retroactive Carbon Pricing (ReCaP), a market mechanism in which polluters offload the payment of ReSCCU adjustments to insurers. To alleviate systematic risks and minimize government involvement, we introduce the Private ReCaP (PReCaP) prediction market, which could see real-world implementation based on the engagement of a few high net-worth individuals or independent institutions.
2021-12-31
SSRN Electronic Journal (unknown)
doi.org
arxiv.org
Question Personalization in an Intelligent Tutoring System
Sabina Elkins
Robert Belfer
Ekaterina Kochmar
Iulian V. Serban
Jackie CK Cheung
2021-12-31
AIED (2) (published)
doi.org
arxiv.org
RAPPPID: towards generalizable protein interaction prediction with AWD-LSTM twin networks
Joseph Szymborski
Amin Emad
Online-only supplementary data is available at the journal’s website.
2021-12-31
Bioinformatics (published)
doi.org
Realistic Evaluation of Transductive Few-Shot Learning - Supplementary Material
Olivier Veilleux
Éts Montréal
Malik Boudiaf
Pablo Piantanida
Ismail Ben
Ayed Éts Montreal
In the main tables of the paper, we did not include the performances of α-TIM in the standard balanced setting. Here, we emphasize that α-… (see more)TIM is a generalization of TIM [1] as when α → 1 (i.e., the α-entropies tend to the Shannon entropies), α-TIM tends to TIM. Therefore, in the standard setting, where optimal hyper-parameter α is obtained over validation tasks that are balanced (as in the standard validation tasks of the original TIM and the other existing methods), the performance of α-TIM is the same as TIM. When α is tuned on balanced validation tasks, we obtain an optimal value of α very close to 1, and our α-mutual information approaches the standard mutual information. When the validation tasks are uniformly random, as in our new setting and in the validation plots we provided in the main figure, one can see that the performance of α-TIM remains competitive when we tend to balanced testing tasks (i.e., when a is increasing), but is significantly better than TIM when we tend to uniformly-random testing tasks (a = 1). These results illustrate the flexibility of α-divergences, and are in line with the technical analysis provided in the main paper.
2021-12-31
(published)
www.semanticscholar.org
Robustness of Whittle Index Policy to Model Approximation
Amit Sinha
Aditya Mahajan
2021-12-31
Social Science Research Network (published)
doi.org
Scalable Operator Allocation for Multirobot Assistance: A Restless Bandit Approach
Abhinav Dahiya
Nima Akbarzadeh
Aditya Mahajan
Stephen L. Smith
In this article, we consider the problem of allocating human operators in a system with multiple semiautonomous robots. Each robot is requir… (see more)ed to perform an independent sequence of tasks, subject to a chance of failing and getting stuck in a fault state at every task. If and when required, a human operator can assist or teleoperate a robot. Conventional dynamic programming-based techniques used to solve such problems face scalability issues due to an exponential growth of state and action spaces with the number of robots and operators. In this article, we derive conditions under which the operator allocation problem satisfies a technical condition called indexability, thereby enabling the use of the Whittle index heuristic. The conditions are easy to check, and we show that they hold for a wide range of problems of interest. Our key insight is to leverage the structure of the value function of individual robots, resulting in conditions that can be verified separately for each state of each robot. We apply these conditions to two types of transitions commonly seen in remote robot supervision systems. Through numerical simulations, we demonstrate the efficacy of Whittle index policy as a near-optimal and scalable approach that outperforms existing scalable methods.
2021-12-31
IEEE Transactions on Control of Network Systems (published)
doi.org
arxiv.org
Self-Supervised Acoustic Anomaly Detection Via Contrastive Learning.
Hadi Hojjati
Narges Armanfard
We propose an acoustic anomaly detection algorithm based on the framework of contrastive learning. Contrastive learning is a recently propos… (see more)ed self-supervised approach that has shown promising results in image classification and speech recognition. However, its application in anomaly detection is underexplored. Earlier studies have demonstrated that it can achieve state-of-the-art performance in image anomaly detection, but its capability in anomalous sound detection is yet to be investigated. For the first time, we propose a contrastive learning-based framework that is suitable for acoustic anomaly detection. Since most existing contrastive learning approaches are targeted toward images, the effect of other data transformations on the performance of the algorithm is unknown. Our framework learns a representation from unlabeled data by applying audio-specific data augmentations. We show that in the resulting latent space, normal and abnormal points are distinguishable. Experiments conducted on the MIMII dataset confirm that our approach can outperform competing methods in detecting anomalies.
2021-12-31
IEEE International Conference on Acoustics, Speech and Signal Processing (unknown)
doi.org
Sharpness-Aware Training for Accurate Inference on Noisy DNN Accelerators
Goncalo Mordido
A. Chandar
Franccois Leduc-Primeau
Energy-efficient deep neural network (DNN) accelerators are prone to non-idealities that degrade DNN performance at inference time. To miti… (see more)gate such degradation, existing methods typically add perturbations to the DNN weights during training to simulate inference on noisy hardware. However, this often requires knowledge about the target hardware and leads to a trade-off between DNN performance and robustness, decreasing the former to increase the latter. In this work, we show that applying sharpness-aware training by optimizing for both the loss value and the loss sharpness significantly improves robustness to noisy hardware at inference time while also increasing DNN performance. We further motivate our results by showing a high correlation between loss sharpness and model robustness. We show superior performance compared to injecting noise during training and aggressive weight clipping on multiple architectures, optimizers, datasets, and training regimes without relying on any assumptions about the target hardware. This is observed on a generic noise model as well as on accurate noise simulations from real hardware.
2021-12-31
arXiv.org (preprint)
doi.org
Single Allocation Hub Location with Heterogeneous Economies of Scale
Borzou Rostami
Masoud Chitsaz
Okan Arslan
Gilbert Laporte
Andrea Lodi
The economies of scale in hub location is usually modeled by a constant parameter, which captures the benefits companies obtain through cons… (see more)olidation. In their article “Single allocation hub location with heterogeneous economies of scale,” Rostami et al. relax this assumption and consider hub-hub connection costs as piecewise linear functions of the flow amounts. This spoils the triangular inequality property of the distance matrix, making the classical flow-based model invalid and further complicates the problem. The authors tackle the challenge by building a mixed-integer quadratically constrained program and by developing a methodology based on constructing Lagrangian function, linear dual functions, and specialized polynomial-time algorithms to generate enhanced cuts. The developed method offers a new strategy in Benders-type decomposition through relaxing a set of complicating constraints in subproblems when such relaxation is tight. The results confirm the efficacy of the solution methods in solving large-scale problem instances.
2021-12-31
Operations Research (published)
doi.org