Publications

Can Ensembling Pre-processing Algorithms Lead to Better Machine Learning Fairness?
Khaled Badran
Pierre-Olivier Côté
Amanda Kolopanis
Rached Bouchoucha
Antonio Collante
Diego Elias Costa
Emad Shihab
Foutse Khomh
As machine learning (ML) systems get adopted in more critical areas, it has become increasingly crucial to address the bias that could occur… (see more) in these systems. Several fairness pre-processing algorithms are available to alleviate implicit biases during model training. These algorithms employ different concepts of fairness, often leading to conflicting strategies with consequential trade-offs between fairness and accuracy. In this work, we evaluate three popular fairness pre-processing algorithms and investigate the potential for combining all algorithms into a more robust pre-processing ensemble. We report on lessons learned that can help practitioners better select fairness algorithms for their models.
2022-12-31
Computer (published)
doi.org
arxiv.org
Can Forward Gradient Match Backpropagation?
Louis Fournier
Stéphane Rivaud
Eugene Belilovsky
Michael Eickenberg
Edouard Oyallon
Forward Gradients - the idea of using directional derivatives in forward differentiation mode - have recently been shown to be utilizable fo… (see more)r neural network training while avoiding problems generally associated with backpropagation gradient computation, such as locking and memorization requirements. The cost is the requirement to guess the step direction, which is hard in high dimensions. While current solutions rely on weighted averages over isotropic guess vector distributions, we propose to strongly bias our gradient guesses in directions that are much more promising, such as feedback obtained from small, local auxiliary networks. For a standard computer vision neural network, we conduct a rigorous study systematically covering a variety of combinations of gradient targets and gradient guesses, including those previously presented in the literature. We find that using gradients obtained from a local loss as a candidate direction drastically improves on random noise in Forward Gradient methods.
2022-12-31
ICML (published)
doi.org
proceedings.mlr.press
Can AI Read the Minds of Corporate Executives?
Nicolas Chapados
Zhenzhen Fan
Ruslan Goyenko
Issam Hadj Laradji
Fred Liu
Chengyu Zhang
2022-12-31
SSRN Electronic Journal (published)
doi.org
Can Workers Meaningfully Consent to Workplace Wellbeing Technologies?
Shreya Chowdhary
Anna Kawakami
Jina Suh
Mary L Gray
A.R. Olteanu
Koustuv Saha
2022-12-31
FAccT (published)
doi.org
arxiv.org
A circulating proteome-informed prognostic model of COVID-19 disease activity that relies on 1 routinely available clinical laboratories 2
William Ma
Antoine Soulé
Karine Tremblay
Simon Rousseau
Amin Emad
Abstract
2022-12-31
(published)
www.semanticscholar.org
Combining Parameter-efficient Modules for Task-level Generalisation
Edoardo M. Ponti
Alessandro Sordoni
Yoshua Bengio
Siva Reddy
A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to ne… (see more)w tasks. In this work, we assume that each task is associated with a subset of latent skills from an (arbitrary size) inventory. In turn, each skill corresponds to a parameter-efficient (sparse / low-rank) model adapter. By jointly learning adapters and a routing function that allocates skills to each task, the full network is instantiated as the average of the parameters of active skills. We propose several inductive biases that encourage re-usage and composition of the skills, including variable-size skill allocation and a dual-speed learning rate. We evaluate our latent-skill model in two main settings: 1) multitask reinforcement learning for instruction following on 8 levels of the BabyAI platform; and 2) few-shot fine-tuning of language models on 160 NLP tasks of the CrossFit benchmark. We find that the modular design of our network enhances sample efficiency in reinforcement learning and few-shot generalisation in supervised learning, compared to a series of baselines. These include models where parameters are fully shared, task-specific, conditionally generated (HyperFormer), or sparse mixture-of-experts (TaskMoE).
2022-12-31
Conference of the European Chapter of the Association for Computational Linguistics (published)
doi.org
Conditional Flow Matching: Simulation-Free Dynamic Optimal Transport
Alexander Tong
Nikolay Malkin
Guillaume Huguet
Yanlei Zhang
Jarrid Rector-Brooks
Kilian Fatras
Guy Wolf
Yoshua Bengio
2022-12-31
arXiv.org (preprint)
doi.org
Constant Memory Attentive Neural Processes
Leo Feng
Frederick Tung
Hossein Hajimirsadeghi
Yoshua Bengio
Mohamed Osama Ahmed
2022-12-31
arXiv.org (preprint)
doi.org
Continually learning representations at scale
Alexandre Galashov
Jovana Mitrovic
Dhruva Tirumala
Yee Whye Teh
Timothy Nguyen
Arslan Chaudhry
Razvan Pascanu
2022-12-31
CoLLAs (published)
proceedings.mlr.press
proceedings.mlr.press
Contrast-agnostic deep learning-based registration pipeline: Validation in spinal cord multimodal MRI data
E. Béal
J. Cohen-Adad
Medical image registration can be challenging, in that optimal solutions depend on the application domain (unimodal, multimodal, intra-subje… (see more)ct and inter-subject), anatomical sites (e.g., brain, spinal cord (SC) and lungs), dimensionality of the data (2D, 3D and 4D), deformation constraints (rigid, affine and nonlinear) and computational time. Solutions that could accommodate a large variety of applications while producing satisfactory results are needed. SynthMorph was recently introduced as an unsupervised deep learning–based registration method. A particularly interesting feature is that training is performed on synthetic data so that registration becomes agnostic to image contrast and anatomy. However, SynthMorph is particularly sensitive to the initial closeness of the images. In this work, we extend the SynthMorph method by developing a cascaded pipeline of two models that can accommodate large and fine deformations, respectively. We also validate this pipeline for the registration of intra-subject multimodal and inter-subject uni/multimodal MRI data of the SC. This task is known to be particularly difficult due to the vicinity of multiple tissue types whose morphometrics can vary substantially across subjects and contrasts. Evaluation of the method was conducted on a publicly available dataset (spine-generic, 267 subjects) and was compared with a state-of-the-art benchmark: Spinal Cord Toolbox and Advanced Normalization Tools. Results demonstrate better registration accuracy compared with the benchmark and about 24–30 times faster on CPUs depending on the image size. This proposed pipeline provides an easy-to-use, accurate and fast solution for multimodal 3D registration. The code and trained models are freely available at https://github.com/ivadomed/multimodal-registration.
2022-12-31
Aperture Neuro (published)
doi.org
Contrasting intra-modal and ranking cross-modal hard negatives to enhance visio-linguistic compositional understanding
Le Zhang
Rabiul Awal
Aishwarya Agrawal
Vision-Language Models (VLMs), such as CLIP, exhibit strong image-text comprehension abilities, facilitating advances in several downstream … (see more)tasks such as zero-shot image classification, image-text retrieval, and text-to-image generation. However, the compositional reasoning abilities of existing VLMs remains subpar. The root of this limitation lies in the inadequate alignment between the images and captions in the pretraining datasets. Additionally, the current contrastive learning objective fails to focus on fine-grained grounding components like relations, actions, and attributes, resulting in "bag-of-words" representations. We introduce a simple and effective method to improve compositional reasoning in VLMs. Our method better leverages available datasets by refining and expanding the standard image-text contrastive learning framework. Our approach does not require specific annotations and does not incur extra parameters. When integrated with CLIP, our technique yields notable improvement over state-of-the-art baselines across five vision-language compositional benchmarks. We open-source our code at https://github.com/lezhang7/Enhance-FineGrained.
2022-12-31
arXiv.org (preprint)
doi.org
arxiv.org
Contrastive Positive Unlabeled Learning
Anish Acharya
Sujay Sanghavi
Li Jing
Bhargav Bhushanam
Michael G. Rabbat
I. Dhillon
Self-supervised pretraining on unlabeled data followed by supervised fine-tuning on labeled data is a popular paradigm for learning from lim… (see more)ited labeled examples. We extend this paradigm to the classical positive unlabeled (PU) setting, where the task is to learn a binary classifier given only a few labeled positive samples, and (often) a large amount of unlabeled samples (which could be positive or negative). We first propose a simple extension of standard infoNCE family of contrastive losses, to the PU setting; and show that this learns superior representations, as compared to existing unsupervised and supervised approaches. We then develop a simple methodology to pseudo-label the unlabeled samples using a new PU-specific clustering scheme; these pseudo-labels can then be used to train the final (positive vs. negative) classifier. Our method handily outperforms state-of-the-art PU methods over several standard PU benchmark datasets, while not requiring a-priori knowledge of any class prior (which is a common assumption in other PU methods). We also provide a simple theoretical analysis that motivates our methods.
2022-12-31
(published)
www.semanticscholar.org