Changjian Shui

Shichun Yang

Boyu Wang

Charles Ling

2024-03-24

Proceedings of the AAAI Conference on Artificial Intelligence (published)

Hessian Aware Low-Rank Weight Perturbation for Continual Learning

Jiaqi Li

Rui Wang

Yuanhao Lai

Sabyasachi Sahoo

Charles Ling

Shichun Yang

Boyu Wang

Fan Zhou

Continual learning aims to learn a series of tasks sequentially without forgetting the knowledge acquired from the previous ones. In this wo… (see more)rk, we propose the Hessian Aware Low-Rank Perturbation algorithm for continual learning. By modeling the parameter transitions along the sequential tasks with the weight matrix transformation, we propose to apply the low-rank approximation on the task-adaptive parameters in each layer of the neural networks. Specifically, we theoretically demonstrate the quantitative relationship between the Hessian and the proposed low-rank approximation. The approximation ranks are then globally determined according to the marginal increment of the empirical loss estimated by the layer-specific gradient and low-rank approximation error. Furthermore, we control the model capacity by pruning less important parameters to diminish the parameter growth. We conduct extensive experiments on various benchmarks, including a dataset with large-scale tasks, and compare our method against some recent state-of-the-art methods to demonstrate the effectiveness and scalability of our proposed method. Empirical results show that our method performs better on different benchmarks, especially in achieving task order robustness and handling the forgetting issue. The source code is at https://github.com/lijiaqi/HALRP.

2024-01-01

IEEE Trans. Knowl. Data Eng. (published)

Hessian Aware Low-Rank Perturbation for Order-Robust Continual Learning

Jiaqi Li

Rui Wang

Yuanhao Lai

Sabyasachi Sahoo

Charles Ling

Shichun Yang

Boyu Wang

Fan Zhou

2023-11-26

ArXiv (preprint)

Hessian Aware Low-Rank Perturbation for Order-Robust Continual Learning

Jiaqi Li

Rui Wang

Yuanhao Lai

Sabyasachi Sahoo

Charles Ling

Shichun Yang

Boyu Wang

Fan Zhou

2023-11-26

ArXiv (preprint)

Mitigating Calibration Bias Without Fixed Attribute Grouping for Improved Fairness in Medical Imaging Analysis

Justin Szeto

Raghav Mehta

Douglas Arnold

2023-10-08

OpenReview.net/Archive (published)

openreview.net

Towards More General Loss and Setting in Unsupervised Domain Adaptation

Ruizhi Pu

Gezheng Xu

Jun Wen

Fan Zhou

Charles Ling

Boyu Wang

In this article, we present an analysis of unsupervised domain adaptation with a series of theoretical and algorithmic results. We derive a … (see more)novel Rényi-

2023-10-01

IEEE Transactions on Knowledge and Data Engineering (published)

Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysis

Raghav Mehta

Although deep learning (DL) models have shown great success in many medical image analysis tasks, deployment of the resulting models into r… (see more)eal clinical contexts requires: (1) that they exhibit robustness and fairness across different sub-populations, and (2) that the confidence in DL model predictions be accurately expressed in the form of uncertainties. Unfortunately, recent studies have indeed shown significant biases in DL models across demographic subgroups (e.g., race, sex, age) in the context of medical image analysis, indicating a lack of fairness in the models. Although several methods have been proposed in the ML literature to mitigate a lack of fairness in DL models, they focus entirely on the absolute performance between groups without considering their effect on uncertainty estimation. In this work, we present the first exploration of the effect of popular fairness models on overcoming biases across subgroups in medical image analysis in terms of bottom-line performance, and their effects on uncertainty quantification. We perform extensive experiments on three different clinically relevant tasks: (i) skin lesion classification, (ii) brain tumour segmentation, and (iii) Alzheimer's disease clinical score regression. Our results indicate that popular ML methods, such as data-balancing and distributionally robust optimization, succeed in mitigating fairness issues in terms of the model performances for some of the tasks. However, this can come at the cost of poor uncertainty estimates associated with the model predictions. This tradeoff must be mitigated if fairness models are to be adopted in medical image analysis.

2023-04-04

MIDL.io/2023/Conference (poster)

openreview.net

Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysis

Raghav Mehta

2023-04-04

MIDL.io/2023/Conference (poster)

openreview.net

Gap Minimization for Knowledge Sharing and Transfer

Boyu Wang

Jorge A. Mendez

Fan Zhou

Di Wu

Gezheng Xu

Eric R. Eaton

Learning from multiple related tasks by knowledge sharing and transfer has become increasingly relevant over the last two decades. In order … (see more)to successfully transfer information from one task to another, it is critical to understand the similarities and differences between the domains. In this paper, we introduce the notion of \emph{performance gap}, an intuitive and novel measure of the distance between learning tasks. Unlike existing measures which are used as tools to bound the difference of expected risks between tasks (e.g.,

Clinically Plausible Pathology-Anatomy Disentanglement in Patient Brain MRI with Structured Variational Priors

Anjun Hu

Jean-Pierre R. Falet

Brennan Nichyporuk

Douglas Arnold

Sotirios A. Tsaftaris

We propose a hierarchically structured variational inference model for accurately disentangling observable evidence of disease (e.g. brain l… (see more)esions or atrophy) from subject-specific anatomy in brain MRIs. With flexible, partially autoregressive priors, our model (1) addresses the subtle and fine-grained dependencies that typically exist between anatomical and pathological generating factors of an MRI to ensure the clinical validity of generated samples; (2) preserves and disentangles finer pathological details pertaining to a patient's disease state. Additionally, we experiment with an alternative training configuration where we provide supervision to a subset of latent units. It is shown that (1) a partially supervised latent space achieves a higher degree of disentanglement between evidence of disease and subject-specific anatomy; (2) when the prior is formulated with an autoregressive structure, knowledge from the supervision can propagate to the unsupervised latent units, resulting in more informative latent representations capable of modelling anatomy-pathology interdependencies.

2022-11-15

ArXiv (preprint)

Lifelong Online Learning from Accumulated Knowledge

William Wang

Ihsen Hedhli

Chi Man Wong

Feng Wan

Boyu Wang

In this article, we formulate lifelong learning as an online transfer learning procedure over consecutive tasks, where learning a given task… (see more) depends on the accumulated knowledge. We propose a novel theoretical principled framework, lifelong online learning, where the learning process for each task is in an incremental manner. Specifically, our framework is composed of two-level predictions: the prediction information that is solely from the current task; and the prediction from the knowledge base by previous tasks. Moreover, this article tackled several fundamental challenges: arbitrary or even non-stationary task generation process, an unknown number of instances in each task, and constructing an efficient accumulated knowledge base. Notably, we provide a provable bound of the proposed algorithm, which offers insights on the how the accumulated knowledge improves the predictions. Finally, empirical evaluations on both synthetic and real datasets validate the effectiveness of the proposed algorithm.

2022-10-17

ACM Transactions on Knowledge Discovery from Data (published)

A novel domain adaptation theory with Jensen-Shannon divergence

Qi CHEN

Jun Wen

Fan Zhou

Boyu Wang

2022-09-01

Knowledge-Based Systems (published)