Publications

Understanding Zero-shot Adversarial Robustness for Large-Scale Models
Chengzhi Mao
Scott Geng
Junfeng Yang
Xin Wang
Carl Vondrick
Pretrained large-scale vision-language models like CLIP have exhibited strong generalization over unseen tasks. Yet imperceptible adversaria… (voir plus)l perturbations can significantly reduce CLIP's performance on new tasks. In this work, we identify and explore the problem of adapting large-scale models for zero-shot adversarial robustness. We first identify two key factors during model adaption--training losses and adaptation methods--that affect the model's zero-shot adversarial robustness. We then propose a text-guided contrastive adversarial training loss, which aligns the text embeddings and the adversarial visual features with contrastive learning on a small set of training data. We apply this training loss to two adaption methods, model finetuning and visual prompt tuning. We find that visual prompt tuning is more effective in the absence of texts, while finetuning wins in the existence of text guidance. Overall, our approach significantly improves the zero-shot adversarial robustness over CLIP, seeing an average improvement of 31 points over ImageNet and 15 zero-shot datasets. We hope this work can shed light on understanding the zero-shot adversarial robustness of large-scale models.
2023-01-31
ICLR.cc/2023/Conference (poster)
doi.org
openreview.net
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?
Mansheej Paul
Feng Chen
Brett W. Larsen
Jonathan Frankle
Surya Ganguli
Gintare Karolina Dziugaite
Modern deep learning involves training costly, highly overparameterized networks, thus motivating the search for sparser networks that can s… (voir plus)till be trained to the same accuracy as the full network (i.e. matching). Iterative magnitude pruning (IMP) is a state of the art algorithm that can find such highly sparse matching subnetworks, known as winning tickets. IMP operates by iterative cycles of training, masking smallest magnitude weights, rewinding back to an early training point, and repeating. Despite its simplicity, the underlying principles for when and how IMP finds winning tickets remain elusive. In particular, what useful information does an IMP mask found at the end of training convey to a rewound network near the beginning of training? How does SGD allow the network to extract this information? And why is iterative pruning needed? We develop answers in terms of the geometry of the error landscape. First, we find that
2023-01-31
ICLR.cc/2023/Conference (notable)
doi.org
openreview.net
Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top
Eduard Gorbunov
Samuel Horváth
Peter Richtárik
Gauthier Gidel
Byzantine-robustness has been gaining a lot of attention due to the growth of the interest in collaborative and federated learning. However,… (voir plus) many fruitful directions, such as the usage of variance reduction for achieving robustness and communication compression for reducing communication costs, remain weakly explored in the field. This work addresses this gap and proposes Byz-VR-MARINA - a new Byzantine-tolerant method with variance reduction and compression. A key message of our paper is that variance reduction is key to fighting Byzantine workers more effectively. At the same time, communication compression is a bonus that makes the process more communication efficient. We derive theoretical convergence guarantees for Byz-VR-MARINA outperforming previous state-of-the-art for general non-convex and Polyak-Lojasiewicz loss functions. Unlike the concurrent Byzantine-robust methods with variance reduction and/or compression, our complexity results are tight and do not rely on restrictive assumptions such as boundedness of the gradients or limited compression. Moreover, we provide the first analysis of a Byzantine-tolerant method supporting non-uniform sampling of stochastic gradients. Numerical experiments corroborate our theoretical findings.
2023-01-31
ICLR.cc/2023/Conference (poster)
doi.org
openreview.net
Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning
John Nguyen
Jianyu Wang
Kshitiz Malik
Maziar Sanjabi
Michael G. Rabbat
AI Meta
2023-01-31
ICLR.cc/2023/Conference (notable)
doi.org
openreview.net
BARVINN: Arbitrary Precision DNN Accelerator Controlled by a RISC-V CPU
Mohammadhossein Askarihemmat
Sean Wagner
Olexa Bilaniuk
Yassine Hariri
Yvon Savaria
Jean-Pierre David
We present a DNN accelerator that allows inference at arbitrary precision with dedicated processing elements that are configurable at the bi… (voir plus)t level. Our DNN accelerator has 8 Processing Elements controlled by a RISC-V controller with a combined 8.2 TMACs of computational power when implemented with the recent Alveo U250 FPGA platform. We develop a code generator tool that ingests CNN models in ONNX format and generates an executable command stream for the RISC-V controller. We demonstrate the scalable throughput of our accelerator by running different DNN kernels and models when different quantization levels are selected. Compared to other low precision accelerators, our accelerator provides run time programmability without hardware reconfiguration and can accelerate DNNs with multiple quantization levels, regardless of the target FPGA size. BARVINN is an open source project and it is available at https://github.com/hossein1387/BARVINN.
2023-01-30
Proceedings of the 28th Asia and South Pacific Design Automation Conference (publié)
doi.org
arxiv.org
Single-cell multi-omic topic embedding reveals cell-type-specific and COVID-19 severity-related immune signatures
Manqi Zhou
Hao Zhang
Zilong Bai
Dylan Mann-Krzisnik
Fei Wang
Yue Li
The advent of single-cell multi-omics sequencing technology makes it possible for re-searchers to leverage multiple modalities for individua… (voir plus)l cells and explore cell heterogeneity. However, the high dimensional, discrete, and sparse nature of the data make the downstream analysis particularly challenging. Most of the existing computational methods for single-cell data analysis are either limited to single modality or lack flexibility and interpretability. In this study, we propose an interpretable deep learning method called multi-omic embedded topic model (moETM) to effectively perform integrative analysis of high-dimensional single-cell multimodal data. moETM integrates multiple omics data via a product-of-experts in the encoder for efficient variational inference and then employs multiple linear decoders to learn the multi-omic signatures of the gene regulatory programs. Through comprehensive experiments on public single-cell transcriptome and chromatin accessibility data (i.e., scRNA+scATAC), as well as scRNA and proteomic data (i.e., CITE-seq), moETM demonstrates superior performance compared with six state-of-the-art single-cell data analysis methods on seven publicly available datasets. By applying moETM to the scRNA+scATAC data in human bone marrow mononuclear cells (BMMCs), we identified sequence motifs corresponding to the transcription factors that regulate immune gene signatures. Applying moETM analysis to CITE-seq data from the COVID-19 patients revealed not only known immune cell-type-specific signatures but also composite multi-omic biomarkers of critical conditions due to COVID-19, thus providing insights from both biological and clinical perspectives.
2023-01-30
bioRxiv (prépublication)
doi.org
Leveraging the Third Dimension in Contrastive Learning
Sumukh K Aithal
Anirudh Goyal
Alex Lamb
Yoshua Bengio
Michael Curtis Mozer
Self-Supervised Learning (SSL) methods operate on unlabeled data to learn robust representations useful for downstream tasks. Most SSL metho… (voir plus)ds rely on augmentations obtained by transforming the 2D image pixel map. These augmentations ignore the fact that biological vision takes place in an immersive three-dimensional, temporally contiguous environment, and that low-level biological vision relies heavily on depth cues. Using a signal provided by a pretrained state-of-the-art monocular RGB-to-depth model (the \emph{Depth Prediction Transformer}, Ranftl et al., 2021), we explore two distinct approaches to incorporating depth signals into the SSL framework. First, we evaluate contrastive learning using an RGB+depth input representation. Second, we use the depth signal to generate novel views from slightly different camera positions, thereby producing a 3D augmentation for contrastive learning. We evaluate these two approaches on three different SSL methods -- BYOL, SimSiam, and SwAV -- using ImageNette (10 class subset of ImageNet), ImageNet-100 and ImageNet-1k datasets. We find that both approaches to incorporating depth signals improve the robustness and generalization of the baseline SSL methods, though the first approach (with depth-channel concatenation) is superior. For instance, BYOL with the additional depth channel leads to an increase in downstream classification accuracy from 85.3\% to 88.0\% on ImageNette and 84.1\% to 87.0\% on ImageNet-C.
2023-01-26
ArXiv (prépublication)
doi.org
openreview.net
Accompanying patients in clinical oncology teams: Reported activities and perceived effects
Marie‐Pascale Pomey
Jesseca Paquette
Monica Iliescu‐Nelea
Cécile Vialaron
Rim Mourad
Karine Bouchard
Louise Normandin
Marie‐Andrée Côté
Mado Desforges
Pénélope Pomey‐Carpentier
Israël Fortin
Isabelle Ganache
Catherine Régis
Zeev Rosberger
Danielle Charpentier
Lynda Bélanger
Michel Dorval
Djahanchah P. Ghadiri
Mélanie Lavoie‐Tremblay
Antoine Boivin … (voir 4 de plus)
Jean‐François Pelletier
Nicolas Fernandez
Alain M. Danino
Michèle de Guise
Two patient–researchers have contributed to the study design, the conduct of the study, the data analysis and interpretation, as well as i… (voir plus)n the preparation and writing of this manuscript.
2023-01-25
Health Expectations (publié)
doi.org
Semi-Supervised Object Detection for Agriculture
Gabriel Tseng
Krisztina Sinkovics
Tom Watsham
David Rolnick
Thomas C. Walters
2023-01-25
AAAI.org/2023/Workshop/AIAFS (poster)
openreview.net
openreview.net
Enhancing Medical Image Segmentation with TransCeption: A Multi-Scale Feature Fusion Approach
Reza Azad
Yiwei Jia
Ehsan Khodapanah Aghdam
Julien Cohen-Adad
Dorit Merhof
While CNN-based methods have been the cornerstone of medical image segmentation due to their promising performance and robustness, they suff… (voir plus)er from limitations in capturing long-range dependencies. Transformer-based approaches are currently prevailing since they enlarge the reception field to model global contextual correlation. To further extract rich representations, some extensions of the U-Net employ multi-scale feature extraction and fusion modules and obtain improved performance. Inspired by this idea, we propose TransCeption for medical image segmentation, a pure transformer-based U-shape network featured by incorporating the inception-like module into the encoder and adopting a contextual bridge for better feature fusion. The design proposed in this work is based on three core principles: (1) The patch merging module in the encoder is redesigned with ResInception Patch Merging (RIPM). Multi-branch transformer (MB transformer) adopts the same number of branches as the outputs of RIPM. Combining the two modules enables the model to capture a multi-scale representation within a single stage. (2) We construct an Intra-stage Feature Fusion (IFF) module following the MB transformer to enhance the aggregation of feature maps from all the branches and particularly focus on the interaction between the different channels of all the scales. (3) In contrast to a bridge that only contains token-wise self-attention, we propose a Dual Transformer Bridge that also includes channel-wise self-attention to exploit correlations between scales at different stages from a dual perspective. Extensive experiments on multi-organ and skin lesion segmentation tasks present the superior performance of TransCeption compared to previous work. The code is publicly available at https://github.com/mindflow-institue/TransCeption.
2023-01-24
ArXiv (prépublication)
doi.org
arxiv.org
Explainable Machine Learning Model to Predict COVID-19 Severity Among Older Adults in the Province of Quebec.
S. A. Rahimi
Charlene H Chu
Roland M. Grad
Mark Karanofsky
Mylene Arsenault
Charlene Esteban Ronquillo
Isabelle Vedel
K. McGilton
Machelle Wilchesky
Context: Patients over the age of 65 years are more likely to experience higher severity and mortality rates than other populations from COV… (voir plus)ID-19. Clinicians need assistance in supporting their decisions regarding the management of these patients. Artificial Intelligence (AI) can help with this regard. However, the lack of explainability-defined as "the ability to understand and evaluate the internal mechanism of the algorithm/computational process in human terms"-of AI is one of the major challenges to its application in health care. We know little about application of explainable AI (XAI) in health care. Objective: In this study, we aimed to evaluate the feasibility of the development of explainable machine learning models to predict COVID-19 severity among older adults. Design: Quantitative machine learning methods. Setting: Long-term care facilities within the province of Quebec. Participants: Patients 65 years and older presented to the hospitals who had a positive polymerase chain reaction test for COVID-19. Intervention: We used XAI-specific methods (e.g., EBM), machine learning methods (i.e., random forest, deep forest, and XGBoost), as well as explainable approaches such as LIME, SHAP, PIMP, and anchor with the mentioned machine learning methods. Outcome measures: Classification accuracy and area under the receiver operating characteristic curve (AUC). Results: The age distribution of the patients (n=986, 54.6% male) was 84.5□19.5 years. The best-performing models (and their performance) were as follows. Deep forest using XAI agnostic methods LIME (97.36% AUC, 91.65 ACC), Anchor (97.36% AUC, 91.65 ACC), and PIMP (96.93% AUC, 91.65 ACC). We found alignment with the identified reasoning of our models' predictions and clinical studies' findings-about the correlation of different variables such as diabetes and dementia, and the severity of COVID-19 in this population. Conclusions: The use of explainable machine learning models, to predict the severity of COVID-19 among older adults is feasible. We obtained a high-performance level as well as explainability in the prediction of COVID-19 severity in this population. Further studies are required to integrate these models into a decision support system to facilitate the management of diseases such as COVID-19 for (primary) health care providers and evaluate their usability among them.
2023-01-23
Big data (publié)
doi.org
Willingness to Engage in Shared Decision Making: Impact of an Educational Intervention for Resident Physicians (SDM-FM)
Roland M. Grad
A. Sandhu
Michael Ferrante
Vinita D'souza
Lily Puterman-salzman
S. A. Rahimi
Gabrielle Stevens
G. Elwyn
2023-01-23
Education and training (publié)
doi.org