Publications

Best-Case Retrieval Evaluation: Improving the Sensitivity of Reciprocal Rank with Lexicographic Precision
Fernando Diaz
Across a variety of ranking tasks, researchers use reciprocal rank to measure the effectiveness for users interested in exactly one relevant… (see more) item. Despite its widespread use, evidence suggests that reciprocal rank is brittle when discriminating between systems. This brittleness, in turn, is compounded in modern evaluation settings where current, high-precision systems may be difficult to distinguish. We address the lack of sensitivity of reciprocal rank by introducing and connecting it to the concept of best-case retrieval, an evaluation method focusing on assessing the quality of a ranking for the most satisfied possible user across possible recall requirements. This perspective allows us to generalize reciprocal rank and define a new preference-based evaluation we call lexicographic precision or lexiprecision. By mathematical construction, we ensure that lexiprecision preserves differences detected by reciprocal rank, while empirically improving sensitivity and robustness across a broad set of retrieval and recommendation tasks.
2022-12-31
EVIA (published)
doi.org
arxiv.org
Bringing language to dynamic brain states: the default network dominates neural responses to evolving movie stories
Enning Yang
Filip Milisav
Jakub Kopal
Avram J. Holmes
Georgios D. Mitsis
Bratislav Misic
Emily S. Finn
Danilo Bzdok
Naturalistic neuroscience opened the door to new insights into neural circuits that serve real-world dynamic perception. Such studies have o… (see more)ften neglected the rich texture of the movie narrative itself, but semantic content can be used to contextualize the induced neural responses. Here, we translated natural language processing tools from machine learning to characterize brain states estimated from hidden Markov models. Our analytical strategy allowed pitting shallow unimodal against the deep associative brain network layers in explaining how semantic content of the movie links to observed neural activity. Pooling information across >53,000 brain image time points watching Forrest Gump, we could show that distinct dynamic brain states capture unique semantic facets along the unfolding movie narrative. The spatiotemporal dynamics of brain states explicitly captured subject-level responses throughout the brain network hierarchy. Across all analyses, the default network was most intimately linked to semantic information integration, and this neural system switched online for longest durations during movie watching. Further, we identified and described two mechanisms of how the default network liaises dynamically with microanatomically defined subregion partners: the amygdala and the hippocampus. Our study thus unlocks the potential of natural language processing to explore neural processes in everyday life situations that engage key aspects of conscious awareness.
2022-12-31
McGill University (published)
doi.org
Bringing language to dynamic brain states: the default network dominates neural responses to evolving movie stories
Enning Yang
Filip Milisav
Jakub Kopal
Avram J. Holmes
Georgios D. Mitsis
Bratislav Misic
Emily S. Finn
Danilo Bzdok
Naturalistic neuroscience opened the door to new insights into neural circuits that serve real-world dynamic perception. Such studies have o… (see more)ften neglected the rich texture of the movie narrative itself, but semantic content can be used to contextualize the induced neural responses. Here, we translated natural language processing tools from machine learning to characterize brain states estimated from hidden Markov models. Our analytical strategy allowed pitting shallow unimodal against the deep associative brain network layers in explaining how semantic content of the movie links to observed neural activity. Pooling information across >53,000 brain image time points watching Forrest Gump, we could show that distinct dynamic brain states capture unique semantic facets along the unfolding movie narrative. The spatiotemporal dynamics of brain states explicitly captured subject-level responses throughout the brain network hierarchy. Across all analyses, the default network was most intimately linked to semantic information integration, and this neural system switched online for longest durations during movie watching. Further, we identified and described two mechanisms of how the default network liaises dynamically with microanatomically defined subregion partners: the amygdala and the hippocampus. Our study thus unlocks the potential of natural language processing to explore neural processes in everyday life situations that engage key aspects of conscious awareness.
2022-12-31
McGill University (published)
doi.org
Can Ensembling Pre-processing Algorithms Lead to Better Machine Learning Fairness?
Khaled Badran
Pierre-Olivier Côté
Amanda Kolopanis
Rached Bouchoucha
Antonio Collante
Diego Elias Costa
Emad Shihab
Foutse Khomh
As machine learning (ML) systems get adopted in more critical areas, it has become increasingly crucial to address the bias that could occur… (see more) in these systems. Several fairness pre-processing algorithms are available to alleviate implicit biases during model training. These algorithms employ different concepts of fairness, often leading to conflicting strategies with consequential trade-offs between fairness and accuracy. In this work, we evaluate three popular fairness pre-processing algorithms and investigate the potential for combining all algorithms into a more robust pre-processing ensemble. We report on lessons learned that can help practitioners better select fairness algorithms for their models.
2022-12-31
Computer (published)
doi.org
arxiv.org
Can AI Read the Minds of Corporate Executives?
Nicolas Chapados
Zhenzhen Fan
Ruslan Goyenko
Issam Hadj Laradji
Fred Liu
Chengyu Zhang
2022-12-31
SSRN Electronic Journal (published)
doi.org
A circulating proteome-informed prognostic model of COVID-19 disease activity that relies on 1 routinely available clinical laboratories 2
William Ma
Antoine Soulé
Karine Tremblay
Simon Rousseau
Amin Emad
Abstract
2022-12-31
(published)
www.semanticscholar.org
Combinatorial Optimization and Reasoning with Graph Neural Networks
Quentin Cappart
Didier Chételat
Elias B. Khalil
Andrea Lodi
Christopher Morris
Petar Veličković
Combinatorial optimization is a well-established area in operations research and computer science. Until recently, its methods have focused … (see more)on solving problem instances in isolation, ignoring that they often stem from related data distributions in practice. However, recent years have seen a surge of interest in using machine learning, especially graph neural networks (GNNs), as a key building block for combinatorial tasks, either directly as solvers or by enhancing exact solvers. The inductive bias of GNNs effectively encodes combinatorial and relational input due to their invariance to permutations and awareness of input sparsity. This paper presents a conceptual review of recent key advancements in this emerging field, aiming at optimization and machine learning researchers.
2022-12-31
Journal of Machine Learning Research (published)
doi.org
arxiv.org
Combining Parameter-efficient Modules for Task-level Generalisation
Edoardo M. Ponti
Alessandro Sordoni
Yoshua Bengio
Siva Reddy
A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to ne… (see more)w tasks. In this work, we assume that each task is associated with a subset of latent skills from an (arbitrary size) inventory. In turn, each skill corresponds to a parameter-efficient (sparse / low-rank) model adapter. By jointly learning adapters and a routing function that allocates skills to each task, the full network is instantiated as the average of the parameters of active skills. We propose several inductive biases that encourage re-usage and composition of the skills, including variable-size skill allocation and a dual-speed learning rate. We evaluate our latent-skill model in two main settings: 1) multitask reinforcement learning for instruction following on 8 levels of the BabyAI platform; and 2) few-shot fine-tuning of language models on 160 NLP tasks of the CrossFit benchmark. We find that the modular design of our network enhances sample efficiency in reinforcement learning and few-shot generalisation in supervised learning, compared to a series of baselines. These include models where parameters are fully shared, task-specific, conditionally generated (HyperFormer), or sparse mixture-of-experts (TaskMoE).
2022-12-31
Conference of the European Chapter of the Association for Computational Linguistics (published)
doi.org
Conditional Flow Matching: Simulation-Free Dynamic Optimal Transport
Alexander Tong
Nikolay Malkin
Guillaume Huguet
Yanlei Zhang
Jarrid Rector-Brooks
Kilian Fatras
Guy Wolf
Yoshua Bengio
2022-12-31
arXiv.org (preprint)
doi.org
Constant Memory Attentive Neural Processes
Leo Feng
Frederick Tung
Hossein Hajimirsadeghi
Yoshua Bengio
Mohamed Osama Ahmed
2022-12-31
arXiv.org (preprint)
doi.org
Contrast-agnostic deep learning-based registration pipeline: Validation in spinal cord multimodal MRI data
E. Béal
J. Cohen-Adad
Medical image registration can be challenging, in that optimal solutions depend on the application domain (unimodal, multimodal, intra-subje… (see more)ct and inter-subject), anatomical sites (e.g., brain, spinal cord (SC) and lungs), dimensionality of the data (2D, 3D and 4D), deformation constraints (rigid, affine and nonlinear) and computational time. Solutions that could accommodate a large variety of applications while producing satisfactory results are needed. SynthMorph was recently introduced as an unsupervised deep learning–based registration method. A particularly interesting feature is that training is performed on synthetic data so that registration becomes agnostic to image contrast and anatomy. However, SynthMorph is particularly sensitive to the initial closeness of the images. In this work, we extend the SynthMorph method by developing a cascaded pipeline of two models that can accommodate large and fine deformations, respectively. We also validate this pipeline for the registration of intra-subject multimodal and inter-subject uni/multimodal MRI data of the SC. This task is known to be particularly difficult due to the vicinity of multiple tissue types whose morphometrics can vary substantially across subjects and contrasts. Evaluation of the method was conducted on a publicly available dataset (spine-generic, 267 subjects) and was compared with a state-of-the-art benchmark: Spinal Cord Toolbox and Advanced Normalization Tools. Results demonstrate better registration accuracy compared with the benchmark and about 24–30 times faster on CPUs depending on the image size. This proposed pipeline provides an easy-to-use, accurate and fast solution for multimodal 3D registration. The code and trained models are freely available at https://github.com/ivadomed/multimodal-registration.
2022-12-31
Aperture Neuro (published)
doi.org
Contrastive Positive Unlabeled Learning
Anish Acharya
Sujay Sanghavi
Li Jing
Bhargav Bhushanam
Michael G. Rabbat
I. Dhillon
Self-supervised pretraining on unlabeled data followed by supervised fine-tuning on labeled data is a popular paradigm for learning from lim… (see more)ited labeled examples. We extend this paradigm to the classical positive unlabeled (PU) setting, where the task is to learn a binary classifier given only a few labeled positive samples, and (often) a large amount of unlabeled samples (which could be positive or negative). We first propose a simple extension of standard infoNCE family of contrastive losses, to the PU setting; and show that this learns superior representations, as compared to existing unsupervised and supervised approaches. We then develop a simple methodology to pseudo-label the unlabeled samples using a new PU-specific clustering scheme; these pseudo-labels can then be used to train the final (positive vs. negative) classifier. Our method handily outperforms state-of-the-art PU methods over several standard PU benchmark datasets, while not requiring a-priori knowledge of any class prior (which is a common assumption in other PU methods). We also provide a simple theoretical analysis that motivates our methods.
2022-12-31
(published)
www.semanticscholar.org