Xue (Steve) Liu

Google Scholar

Sam Chen

Doctorat - McGill University

can.chen@mila.quebec

dheeraj.vattikonda@mila.quebec

Chengming Hu

Doctorat - McGill University

chengming.hu@mila.quebec

Dheeraj Vattikonda

Maîtrise recherche - McGill University

Dun Yuan

Maîtrise recherche - McGill University

dun.yuan@mila.quebec

Gaspard Zhao

Postdoctorat - McGill University

hanqing.zhao@mila.quebec

Mark Bai

Maîtrise recherche - McGill University

mark.bai@mila.quebec

Haolun Wu

Doctorat - McGill University

haolun.wu@mila.quebec

Junliang Luo

Doctorat - McGill University

junliang.luo@mila.quebec

rishabh.thaney@mila.quebec

Lyu Fuyuan

Doctorat - McGill University

fuyuan.lyu@mila.quebec

Maîtrise recherche - McGill University

shaoxiang.qin@mila.quebec

Shaoxiang Qin

Doctorat - McGill University

Shuhao Zheng

Doctorat - McGill University

shuhao.zheng@mila.quebec

Xuan Li

Doctorat - McGill University

xuan.li@mila.quebec

Ye Yuan

Doctorat - McGill University

Yili Jin

Doctorat - McGill University

yili.jin@mila.quebec

Google Scholar

Ralph Cui

Postdoctorat

yufei.cui@mila.quebec

Apprentissage bidirectionnel pour l’optimisation hors ligne basée sur un modèle

Google Scholar

Zonglun Li

Doctorat - McGill University

zonglun.li@mila.quebec

Billets de blogue

Bidirectional Learning for Offline Model-based Optimization

20 septembre 2023

par

Can Chen

Yingxue Zhang

Xue Liu

Mark Coates

Lire l'article

Publications

Variational Nested Dropout

Yufei Cui

Yu Mao

Ziquan Liu

Qiao Li

Antoni B. Chan

Tei-Wei Kuo

Chun Jason Xue

Nested dropout is a variant of dropout operation that is able to order network parameters or features based on the pre-defined importance du… (voir plus)ring training. It has been explored for: I. Constructing nested nets Cui et al. 2020, Cui et al. 2021: the nested nets are neural networks whose architectures can be adjusted instantly during testing time, e.g., based on computational constraints. The nested dropout implicitly ranks the network parameters, generating a set of sub-networks such that any smaller sub-network forms the basis of a larger one. II. Learning ordered representation Rippel et al. 2014: the nested dropout applied to the latent representation of a generative model (e.g., auto-encoder) ranks the features, enforcing explicit order of the dense representation over dimensions. However, the dropout rate is fixed as a hyper-parameter during the whole training process. For nested nets, when network parameters are removed, the performance decays in a human-specified trajectory rather than in a trajectory learned from data. For generative models, the importance of features is specified as a constant vector, restraining the flexibility of representation learning. To address the problem, we focus on the probabilistic counterpart of the nested dropout. We propose a variational nested dropout (VND) operation that draws samples of multi-dimensional ordered masks at a low cost, providing useful gradients to the parameters of nested dropout. Based on this approach, we design a Bayesian nested neural network that learns the order knowledge of the parameter distributions. We further exploit the VND under different generative models for learning ordered latent distributions. In experiments, we show that the proposed approach outperforms the nested network in terms of accuracy, calibration, and out-of-domain detection in classification tasks. It also outperforms the related generative models on data generation tasks.

2023-08-01

IEEE Transactions on Pattern Analysis and Machine Intelligence (publié)

Bidirectional Learning for Offline Model-based Biological Sequence Design

Can Chen

Yingxue Zhang

Mark Coates

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (publié)

openreview.net

Reinforcement Learning-Based Adaptive Feature Boosting for Smart Grid Intrusion Detection

Chengming Hu

Jun Yan

Intrusion detection systems (IDSs) are crucial in the security monitoring for the smart grid with increasing machine-to-machine communicatio… (voir plus)ns and cyber threats thereafter. However, the multi-sourced, correlated, and heterogeneous smart grid data pose significant challenges to the accurate attack detection by IDSs. To improve the attack detection, this paper proposes Reinforcement Learning-based Adaptive Feature Boosting, which aims to leverage a series of AutoEncoders (AEs) to capture critical features from the multi-sourced smart grid data for the classification of normal, fault, and attack events. Multiple AEs are utilized to extract representative features from different feature sets that are automatically generated through a weighted feature sampling process; each AE-extracted feature set is then applied to build a Random Forest (RF) base classifier. In the feature sampling process, Deep Deterministic Policy Gradient (DDPG) is introduced to dynamically determine the feature sampling probability based on the classification accuracy. The critical features that improve the classification accuracy are assigned larger sampling probabilities and increasingly participate in the training of next AE. The presence of critical features is increased in the event classification over the multi-sourced smart grid data. Considering potential different alarms among base classifiers, an ensemble classifier is further built to distinguish normal, fault, and attack events. Our proposed approach is evaluated on the two realistic datasets collected from Hardware-In-the-Loop (HIL) and WUSTIL-IIOT-2021 security testbeds, respectively. The evaluation on the HIL security dataset shows that our proposed approach achieves the classification accuracy with 97.28%, an effective 5.5% increase over the vanilla Adaptive Feature Boosting. Moreover, the proposed approach not only accurately and stably selects critical features on the WUSTIL-IIOT-2021 dataset based on the significant difference of feature sampling probabilities between critical and uncritical features, i.e., the probabilities greater than 0.08 and less than 0.01, but also outperforms the other best-performing approaches with the increasing Matthew Correlation Coefficient (MCC) of 8.03%.

2023-07-01

IEEE Transactions on Smart Grid (publié)

CATS: A Computation-Aware Transaction Processing System with Proactive Unlocking

Bolun Zhu

Yu Hua

Ziyin Long

With the increasing complexity of network applications and high demands for QoS, transaction processing systems have received more attention… (voir plus)s due to salient features of simplicity and atomicity. Computation operations play an important role in transaction processing systems. However, conventional QoS-based mechanisms become inefficient due to the limited concurrent support upon computation operations, thus causing high time consumption in the critical path of concurrency control. In order to efficiently offer concurrent computations, we propose CATS, a Computation Aware Transaction processing System, to mitigate performance impacts caused by computation operations. CATS further leverages program semantics to defer the execution of transaction operations in the commit phase to alleviate unnecessary conflicts caused by computations. Extensive evaluation results demonstrate that CATS significantly outperforms state-of-the-art designs, including 2PL and OCC based transaction processing systems on high-contended and computation-intensive workloads. We have released the open-source codes in GitHub for public use.

2023-06-19

2023 IEEE/ACM 31st International Symposium on Quality of Service (IWQoS) (publié)

CATS: A Computation-Aware Transaction Processing System with Proactive Unlocking

Bolun Zhu

Yu Hua

Ziyin Long

2023-06-19

International Workshop on Quality of Service (published)

AI Clinics on Mobile (AICOM): Universal AI Doctors for the Underserved and Hard-to-Reach

Tianyi Yang

Tianze Yang

Na An

Ao Kong

Shaoshan Liu

2023-06-17

ArXiv (prépublication)

ANSEL Photobot: A Robot Event Photographer with Semantic Intelligence

Dmitriy Rivkin

Nikhil Kakodkar

David Meger

Oliver Limoyo

Francois Hogan

Our work examines the way in which large language models can be used for robotic planning and sampling in the context of automated photograp… (voir plus)hic documentation. Specifically, we illustrate how to produce a photo-taking robot with an exceptional level of semantic awareness by leveraging recent advances in general purpose language (LM) and vision-language (VLM) models. Given a high-level description of an event we use an LM to generate a natural-language list of photo descriptions that one would expect a photographer to capture at the event. We then use a VLM to identify the best matches to these descriptions in the robot's video stream. The photo portfolios generated by our method are consistently rated as more appropriate to the event by human evaluators than those generated by existing methods.

2023-06-02

2023 IEEE International Conference on Robotics and Automation (ICRA) (publié)

Mixed-Variable PSO with Fairness on Multi-Objective Field Data Replication in Wireless Networks

Dun Yuan

Yujin Nam

Amal Feriani

Abhisek Konar

Di Wu

Seowoo Jang

Digital twins have shown a great potential in supporting the development of wireless networks. They are virtual representations of 5G/6G sys… (voir plus)tems enabling the design of machine learning and optimization-based techniques. Field data replication is one of the critical aspects of building a simulation-based twin, where the objective is to calibrate the simulation to match field performance measurements. Since wireless networks involve a variety of key performance indicators (KPIs), the replication process becomes a multi-objective optimization problem in which the purpose is to minimize the error between the simulated and field data KPIs. Unlike previous works, we focus on designing a data-driven search method to calibrate the simulator and achieve accurate and reliable reproduction of field performance. This work proposes a search-based algorithm based on mixed-variable particle swarm optimization (PSO) to find the optimal simulation parameters. Furthermore, we extend this solution to account for potential conflicts between the KPIs using a-fairness concept to adjust the importance attributed to each KPI during the search. Experiments on field data showcase the effectiveness of our approach to (i) improve the accuracy of the replication, (ii) enhance the fairness between the different KPIs, and (iii) guarantee faster convergence compared to other methods.

2023-06-01

ICC 2023 - IEEE International Conference on Communications (publié)

Multi-Agent Attention Actor-Critic Algorithm for Load Balancing in Cellular Networks

Jikun Kang

Di Wu

Ju Wang

Ekram Hossain

In cellular networks, User Equipment (UE) handoff from one Base Station (BS) to another, giving rise to the load balancing problem among the… (voir plus) BSs. To address this problem, BSs can work collaboratively to deliver a smooth migration (or handoff) and satisfy the UEs' service requirements. This paper formulates the load balancing problem as a Markov game and proposes a Robust Multi-agent Attention Actor-Critic (Robust-MA3C) algorithm that can facilitate collaboration among the BSs (i.e., agents). In particular, to solve the Markov game and find a Nash equilibrium policy, we embrace the idea of adopting a nature agent to model the system uncertainty. Moreover, we utilize the self-attention mechanism, which encourages high-performance BSs to assist low-performance BSs. In addition, we consider two types of schemes, which can facilitate load balancing for both active UEs and idle UEs. We carry out extensive evaluations by simulations, and simulation results illustrate that, compared to the state-of-the-art MARL methods, Robust-MA3C scheme can improve the overall performance by up to 45%.

2023-06-01

ICC 2023 - IEEE International Conference on Communications (publié)

Policy Reuse for Communication Load Balancing in Unseen Traffic Scenarios

Yi Tian Xu

Jimmy Li

Di Wu

M. Jenkin

Seowoo Jang

With the continuous growth in communication network complexity and traffic volume, communication load balancing solutions are receiving incr… (voir plus)easing attention. Specifically, reinforcement learning (RL)-based methods have shown impressive performance compared with traditional rule-based methods. However, standard RL methods generally require an enormous amount of data to train, and generalize poorly to scenarios that are not encountered during training. We propose a policy reuse framework in which a policy selector chooses the most suitable pre-trained RL policy to execute based on the current traffic condition. Our method hinges on a policy bank composed of policies trained on a diverse set of traffic scenarios. When deploying to an unknown traffic scenario, we select a policy from the policy bank based on the similarity between the previous-day traffic of the current scenario and the traffic observed during training. Experiments demonstrate that this framework can outperform classical and adaptive rule-based methods by a large margin.

2023-06-01

ICC 2023 - IEEE International Conference on Communications (publié)

Self-Supervised Transformer Architecture for Change Detection in Radio Access Networks

Igor Kozlov

Dmitriy Rivkin

Wei-Di Chang

Di Wu

Radio Access Networks (RANs) for telecommunications represent large agglomerations of interconnected hardware consisting of hundreds of thou… (voir plus)sands of transmitting devices (cells). Such networks undergo frequent and often heterogeneous changes caused by network operators, who are seeking to tune their system parameters for optimal performance. The effects of such changes are challenging to predict and will become even more so with the adoption of fifth-generation/sixth-generation (5G/6G) networks. Therefore, RAN monitoring is vital for network operators. We propose a self-supervised learning framework that leverages self-attention and self-distillation for this task. It works by detecting changes in Performance Measurement data, a collection of time-varying metrics which reflect a set of diverse measurements of the network performance at the cell level. Experimental results show that our approach outperforms the state of the art by 4% on a real-world based dataset consisting of about hundred thousands time series. It also has the merits of being scalable and generalizable. This allows it to provide deep insight into the specifics of mode of operation changes while relying minimally on expert knowledge.

2023-06-01

ICC 2023 - IEEE International Conference on Communications (publié)

Reinforcement learning for communication load balancing: approaches and challenges

Di Wu

Jimmy Li

Amal Ferini

Yi Tian Xu

M. Jenkin

Seowoo Jang

The amount of cellular communication network traffic has increased dramatically in recent years, and this increase has led to a demand for e… (voir plus)nhanced network performance. Communication load balancing aims to balance the load across available network resources and thus improve the quality of service for network users. Most existing load balancing algorithms are manually designed and tuned rule-based methods where near-optimality is almost impossible to achieve. Furthermore, rule-based methods are difficult to adapt to quickly changing traffic patterns in real-world environments. Reinforcement learning (RL) algorithms, especially deep reinforcement learning algorithms, have achieved impressive successes in many application domains and offer the potential of good adaptabiity to dynamic changes in network load patterns. This survey presents a systematic overview of RL-based communication load-balancing methods and discusses related challenges and opportunities. We first provide an introduction to the load balancing problem and to RL from fundamental concepts to advanced models. Then, we review RL approaches that address emerging communication load balancing issues important to next generation networks, including 5G and beyond. Finally, we highlight important challenges, open issues, and future research directions for applying RL for communication load balancing.

2023-05-31

Frontiers of Computer Science (publié)