Publications

Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning

John Nguyen

Jianyu Wang

Kshitiz Malik

Maziar Sanjabi

Michael G. Rabbat

AI Meta

2023-01-31

ICLR.cc/2023/Conference (notable)

BARVINN: Arbitrary Precision DNN Accelerator Controlled by a RISC-V CPU

Mohammadhossein Askarihemmat

Sean Wagner

Olexa Bilaniuk

Yassine Hariri

Yvon Savaria

Jean-Pierre David

We present a DNN accelerator that allows inference at arbitrary precision with dedicated processing elements that are configurable at the bi… (see more)t level. Our DNN accelerator has 8 Processing Elements controlled by a RISC-V controller with a combined 8.2 TMACs of computational power when implemented with the recent Alveo U250 FPGA platform. We develop a code generator tool that ingests CNN models in ONNX format and generates an executable command stream for the RISC-V controller. We demonstrate the scalable throughput of our accelerator by running different DNN kernels and models when different quantization levels are selected. Compared to other low precision accelerators, our accelerator provides run time programmability without hardware reconfiguration and can accelerate DNNs with multiple quantization levels, regardless of the target FPGA size. BARVINN is an open source project and it is available at https://github.com/hossein1387/BARVINN.

2023-01-30

Proceedings of the 28th Asia and South Pacific Design Automation Conference (published)

Graph Anomaly Detection in Time Series: A Survey

Thi Kieu Khanh Ho

Ali Karami

Narges Armanfard

With the recent advances in technology, a wide range of systems continue to collect a large amount of data over time and thus generate time … (see more)series. Time-Series Anomaly Detection (TSAD) is an important task in various time-series applications such as e-commerce, cybersecurity, vehicle maintenance, and healthcare monitoring. However, this task is very challenging as it requires considering both the intra-variable dependency (relationships within a variable over time) and the inter-variable dependency (relationships between multiple variables) existing in time-series data. Recent graph-based approaches have made impressive progress in tackling the challenges of this field. In this survey, we conduct a comprehensive and up-to-date review of TSAD using graphs, referred to as G-TSAD. First, we explore the significant potential of graph representation for time-series data and and its contributions to facilitating anomaly detection. Then, we review state-of-the-art graph anomaly detection techniques, mostly leveraging deep learning architectures, in the context of time series. For each method, we discuss its strengths, limitations, and the specific applications where it excels. Finally, we address both the technical and application challenges currently facing the field, and suggest potential future directions for advancing research and improving practical outcomes.

2023-01-30

ArXiv (preprint)

Single-cell multi-omic topic embedding reveals cell-type-specific and COVID-19 severity-related immune signatures

Manqi Zhou

Hao Zhang

Zilong Bai

Dylan Mann-Krzisnik

Fei Wang

Yue Li

The advent of single-cell multi-omics sequencing technology makes it possible for re-searchers to leverage multiple modalities for individua… (see more)l cells and explore cell heterogeneity. However, the high dimensional, discrete, and sparse nature of the data make the downstream analysis particularly challenging. Most of the existing computational methods for single-cell data analysis are either limited to single modality or lack flexibility and interpretability. In this study, we propose an interpretable deep learning method called multi-omic embedded topic model (moETM) to effectively perform integrative analysis of high-dimensional single-cell multimodal data. moETM integrates multiple omics data via a product-of-experts in the encoder for efficient variational inference and then employs multiple linear decoders to learn the multi-omic signatures of the gene regulatory programs. Through comprehensive experiments on public single-cell transcriptome and chromatin accessibility data (i.e., scRNA+scATAC), as well as scRNA and proteomic data (i.e., CITE-seq), moETM demonstrates superior performance compared with six state-of-the-art single-cell data analysis methods on seven publicly available datasets. By applying moETM to the scRNA+scATAC data in human bone marrow mononuclear cells (BMMCs), we identified sequence motifs corresponding to the transcription factors that regulate immune gene signatures. Applying moETM analysis to CITE-seq data from the COVID-19 patients revealed not only known immune cell-type-specific signatures but also composite multi-omic biomarkers of critical conditions due to COVID-19, thus providing insights from both biological and clinical perspectives.

2023-01-30

bioRxiv (preprint)

Technical Note—Risk-Averse Regret Minimization in Multistage Stochastic Programs

Mehran Poursoltani

Erick Delage

Angelos Georghiou

2023-01-29

Operational Research (published)

Leveraging the Third Dimension in Contrastive Learning

Sumukh K Aithal

Anirudh Goyal

Alex Lamb

Yoshua Bengio

Michael Curtis Mozer

Self-Supervised Learning (SSL) methods operate on unlabeled data to learn robust representations useful for downstream tasks. Most SSL metho… (see more)ds rely on augmentations obtained by transforming the 2D image pixel map. These augmentations ignore the fact that biological vision takes place in an immersive three-dimensional, temporally contiguous environment, and that low-level biological vision relies heavily on depth cues. Using a signal provided by a pretrained state-of-the-art monocular RGB-to-depth model (the \emph{Depth Prediction Transformer}, Ranftl et al., 2021), we explore two distinct approaches to incorporating depth signals into the SSL framework. First, we evaluate contrastive learning using an RGB+depth input representation. Second, we use the depth signal to generate novel views from slightly different camera positions, thereby producing a 3D augmentation for contrastive learning. We evaluate these two approaches on three different SSL methods -- BYOL, SimSiam, and SwAV -- using ImageNette (10 class subset of ImageNet), ImageNet-100 and ImageNet-1k datasets. We find that both approaches to incorporating depth signals improve the robustness and generalization of the baseline SSL methods, though the first approach (with depth-channel concatenation) is superior. For instance, BYOL with the additional depth channel leads to an increase in downstream classification accuracy from 85.3\% to 88.0\% on ImageNette and 84.1\% to 87.0\% on ImageNet-C.

2023-01-26

ArXiv (preprint)

Accompanying patients in clinical oncology teams: Reported activities and perceived effects

Marie‐Pascale Pomey

Jesseca Paquette

Monica Iliescu‐Nelea

Cécile Vialaron

Rim Mourad

Karine Bouchard

Louise Normandin

Marie‐Andrée Côté

Mado Desforges

Pénélope Pomey‐Carpentier

Israël Fortin

Isabelle Ganache

Catherine Régis

Zeev Rosberger

Danielle Charpentier

Lynda Bélanger

Michel Dorval

Djahanchah P. Ghadiri

Mélanie Lavoie‐Tremblay

Antoine Boivin … (see 4 more)

Jean‐François Pelletier

Nicolas Fernandez

Alain M. Danino

Michèle de Guise

Two patient–researchers have contributed to the study design, the conduct of the study, the data analysis and interpretation, as well as i… (see more)n the preparation and writing of this manuscript.

2023-01-25

Health Expectations (published)

Semi-Supervised Object Detection for Agriculture

Gabriel Tseng

Krisztina Sinkovics

Tom Watsham

David Rolnick

Thomas C. Walters

2023-01-25

AAAI.org/2023/Workshop/AIAFS (poster)

Enhancing Medical Image Segmentation with TransCeption: A Multi-Scale Feature Fusion Approach

Reza Azad

Yiwei Jia

Ehsan Khodapanah Aghdam

Julien Cohen-Adad

Dorit Merhof

While CNN-based methods have been the cornerstone of medical image segmentation due to their promising performance and robustness, they suff… (see more)er from limitations in capturing long-range dependencies. Transformer-based approaches are currently prevailing since they enlarge the reception field to model global contextual correlation. To further extract rich representations, some extensions of the U-Net employ multi-scale feature extraction and fusion modules and obtain improved performance. Inspired by this idea, we propose TransCeption for medical image segmentation, a pure transformer-based U-shape network featured by incorporating the inception-like module into the encoder and adopting a contextual bridge for better feature fusion. The design proposed in this work is based on three core principles: (1) The patch merging module in the encoder is redesigned with ResInception Patch Merging (RIPM). Multi-branch transformer (MB transformer) adopts the same number of branches as the outputs of RIPM. Combining the two modules enables the model to capture a multi-scale representation within a single stage. (2) We construct an Intra-stage Feature Fusion (IFF) module following the MB transformer to enhance the aggregation of feature maps from all the branches and particularly focus on the interaction between the different channels of all the scales. (3) In contrast to a bridge that only contains token-wise self-attention, we propose a Dual Transformer Bridge that also includes channel-wise self-attention to exploit correlations between scales at different stages from a dual perspective. Extensive experiments on multi-organ and skin lesion segmentation tasks present the superior performance of TransCeption compared to previous work. The code is publicly available at https://github.com/mindflow-institue/TransCeption.

2023-01-24

ArXiv (preprint)

Explainable Machine Learning Model to Predict COVID-19 Severity Among Older Adults in the Province of Quebec.

S. A. Rahimi

Charlene H Chu

Roland M. Grad

Mark Karanofsky

Mylene Arsenault

Charlene Esteban Ronquillo

Isabelle Vedel

K. McGilton

Machelle Wilchesky

Context: Patients over the age of 65 years are more likely to experience higher severity and mortality rates than other populations from COV… (see more)ID-19. Clinicians need assistance in supporting their decisions regarding the management of these patients. Artificial Intelligence (AI) can help with this regard. However, the lack of explainability-defined as "the ability to understand and evaluate the internal mechanism of the algorithm/computational process in human terms"-of AI is one of the major challenges to its application in health care. We know little about application of explainable AI (XAI) in health care. Objective: In this study, we aimed to evaluate the feasibility of the development of explainable machine learning models to predict COVID-19 severity among older adults. Design: Quantitative machine learning methods. Setting: Long-term care facilities within the province of Quebec. Participants: Patients 65 years and older presented to the hospitals who had a positive polymerase chain reaction test for COVID-19. Intervention: We used XAI-specific methods (e.g., EBM), machine learning methods (i.e., random forest, deep forest, and XGBoost), as well as explainable approaches such as LIME, SHAP, PIMP, and anchor with the mentioned machine learning methods. Outcome measures: Classification accuracy and area under the receiver operating characteristic curve (AUC). Results: The age distribution of the patients (n=986, 54.6% male) was 84.5□19.5 years. The best-performing models (and their performance) were as follows. Deep forest using XAI agnostic methods LIME (97.36% AUC, 91.65 ACC), Anchor (97.36% AUC, 91.65 ACC), and PIMP (96.93% AUC, 91.65 ACC). We found alignment with the identified reasoning of our models' predictions and clinical studies' findings-about the correlation of different variables such as diabetes and dementia, and the severity of COVID-19 in this population. Conclusions: The use of explainable machine learning models, to predict the severity of COVID-19 among older adults is feasible. We obtained a high-performance level as well as explainability in the prediction of COVID-19 severity in this population. Further studies are required to integrate these models into a decision support system to facilitate the management of diseases such as COVID-19 for (primary) health care providers and evaluate their usability among them.

2023-01-23

Big data (published)

Regeneration Learning: A Learning Paradigm for Data Generation

Xu Tan

Tao Qin

Jiang Bian

Tie-Yan Liu

Yoshua Bengio

2023-01-20

ArXiv (preprint)