Publications

Knowledge Distillation for Federated Learning: a Practical Guide

Alessio Mora

Irene Tenison

Paolo Bellavista

Federated Learning (FL) enables the training of Deep Learning models without centrally collecting possibly sensitive raw data. This paves th… (see more)e way for stronger privacy guarantees when building predictive models. The most used algorithms for FL are parameter-averaging based schemes (e.g., Federated Averaging) that, however, have well known limits: (i) Clients must implement the same model architecture; (ii) Transmitting model weights and model updates implies high communication cost, which scales up with the number of model parameters; (iii) In presence of non-IID data distributions, parameter-averaging aggregation schemes perform poorly due to client model drifts. Federated adaptations of regular Knowledge Distillation (KD) can solve and/or mitigate the weaknesses of parameter-averaging FL algorithms while possibly introducing other trade-offs. In this article, we provide a review of KD-based algorithms tailored for specific FL issues.

2022-11-09

ArXiv (preprint)

doi.org

arxiv.org

A debriefing tool to acquire non-technical skills in trauma courses

Fabio Botelho

Jason M. Harley

Natalie Yanchar

Simone Abib

Ilana Bank

Dan Poenaru

2022-11-07

Surgery Open Science (published)

doi.org

Dissociable brain structural asymmetry patterns reveal unique phenome-wide profiles

Karin Saltoun

Ralph Adolphs

Lynn Paul

Vaibhav Sharma

Joern Diedrichsen

B.T. Thomas Yeo

Danilo Bzdok

2022-11-07

Nature Human Behaviour (published)

doi.org

Multi-Head Adapter Routing for Cross-Task Generalization

Lucas Caccia

Edoardo Ponti

Zhan Su

Matheus Pereira

Nicolas Le Roux

Alessandro Sordoni

Parameter-efficient fine-tuning (PEFT) for cross-task generalization consists in pre-training adapters on a multi-task training set before f… (see more)ew-shot adaptation to test tasks. Polytropon [Ponti et al., 2023] (

2022-11-07

ArXiv (preprint)

arxiv.org

Object-centric causal representation learning

Amin Mansouri

Jason Hartford

Kartik Ahuja

Yoshua Bengio

2022-11-07

NeurIPS.cc/2022/Workshop/NeurReps (poster)

openreview.net

PaReco: patched clones and missed patches among the divergent variants of a software family

Poedjadevie Kadjel Ramkisoen

John Businge

Brent van Bladel

Alexandre Decan

Serge Demeyer

Coen De Roover

Foutse Khomh

Re-using whole repositories as a starting point for new projects is often done by maintaining a variant fork parallel to the original. Howev… (see more)er, the common artifacts between both are not always kept up to date. As a result, patches are not optimally integrated across the two repositories, which may lead to sub-optimal maintenance between the variant and the original project. A bug existing in both repositories can be patched in one but not the other (we see this as a missed opportunity) or it can be manually patched in both probably by different developers (we see this as effort duplication). In this paper we present a tool (named PaReCo) which relies on clone detection to mine cases of missed opportunity and effort duplication from a pool of patches. We analyzed 364 (source to target) variant pairs with 8,323 patches resulting in a curated dataset containing 1,116 cases of effort duplication and 1,008 cases of missed opportunities. We achieve a precision of 91%, recall of 80%, accuracy of 88%, and F1-score of 85%. Furthermore, we investigated the time interval between patches and found out that, on average, missed patches in the target variants have been introduced in the source variants 52 weeks earlier. Consequently, PaReCo can be used to manage variability in “time” by automatically identifying interesting patches in later project releases to be backported to supported earlier releases.

2022-11-07

ESEC/SIGSOFT FSE (published)

doi.org

Posterior samples of source galaxies in strong gravitational lenses with score-based priors

Alexandre Adam

Adam Coogan

Nikolay Malkin

Ronan Legin

Laurence Perreault-Levasseur

Yashar Hezaveh

Yoshua Bengio

2022-11-07

ArXiv (preprint)

doi.org

arxiv.org

Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayes

Mizu Nishikawa-Toomey

Tristan Deleu

Jithendaraa Subramanian

Yoshua Bengio

Laurent Charlin

Bayesian causal structure learning aims to learn a posterior distribution over directed acyclic graphs (DAGs), and the mechanisms that defin… (see more)e the relationship between parent and child variables. By taking a Bayesian approach, it is possible to reason about the uncertainty of the causal model. The notion of modelling the uncertainty over models is particularly crucial for causal structure learning since the model could be unidentifiable when given only a finite amount of observational data. In this paper, we introduce a novel method to jointly learn the structure and mechanisms of the causal model using Variational Bayes, which we call Variational Bayes-DAG-GFlowNet (VBG). We extend the method of Bayesian causal structure learning using GFlowNets to learn not only the posterior distribution over the structure, but also the parameters of a linear-Gaussian model. Our results on simulated data suggest that VBG is competitive against several baselines in modelling the posterior over DAGs and mechanisms, while offering several advantages over existing methods, including the guarantee to sample acyclic graphs, and the flexibility to generalize to non-linear causal mechanisms.

2022-11-04

ArXiv (preprint)

doi.org

arxiv.org

Existing eHealth Solutions for Older Adults Living With Neurocognitive Disorders (Mild and Major) or Dementia and Their Informal Caregivers: Protocol for an Environmental Scan

Ambily Jose

Maxime Sasseville

Samantha Dequanter

Ellen Gorus

Anik Giguère

Anne Bourbonnais

Samira Abbasgholizadeh-Rahimi

Ronald Buyl

Marie-Pierre Gagnon

Background Dementia is one of the main public health priorities for current and future societies worldwide. Over the past years, eHealth sol… (see more)utions have added numerous promising solutions to enhance the health and wellness of people living with dementia-related cognitive problems and their primary caregivers. Previous studies have shown that an environmental scan identifies the knowledge-to-action gap meaningfully. This paper presents the protocol of an environmental scan to monitor the currently available eHealth solutions targeting dementia and other neurocognitive disorders against selected attributes. Objective This study aims to identify the characteristics of currently available eHealth solutions recommended for older adults with cognitive problems and their informal caregivers. To inform the recommendations regarding eHealth solutions for these people, it is important to obtain a comprehensive view of currently available technologies and document their outcomes and conditions of success. Methods We will perform an environmental scan of available eHealth solutions for older adults with cognitive impairment or dementia and their informal caregivers. Potential solutions will be initially identified from a previous systematic review. We will also conduct targeted searches for gray literature on Google and specialized websites covering the regions of Canada and Europe. Technological tools will be scanned based on a preformatted extraction grid. The relevance and efficiency based on the selected attributes will be assessed. Results We will prioritize relevant solutions based on the needs and preferences identified from a qualitative study among older adults with cognitive impairment or dementia and their informal caregivers. Conclusions This environmental scan will identify eHealth solutions that are currently available and scientifically appraised for older adults with cognitive impairment or dementia and their informal caregivers. This knowledge will inform the development of a decision support tool to assist older adults and their informal caregivers in their search for adequate eHealth solutions according to their needs and preferences based on trustable information. International Registered Report Identifier (IRRID) DERR1-10.2196/41015

2022-11-04

JMIR Research Protocols (published)

doi.org

Spectral Regularization: an Inductive Bias for Sequence Modeling

Kaiwen Hou

Guillaume Rabusseau

Hou Rabusseau

2022-11-04

ArXiv (preprint)

doi.org

arxiv.org

Adult neurogenesis acts as a neural regularizer