Xue (Steve) Liu

Jiaxuan Chen

PhD - McGill University

Principal supervisor :

Oana Balmau

Ralph Cui

Postdoctorate

Linfeng Du

PhD - McGill University

Lyu Fuyuan

PhD - McGill University

Co-supervisor :

Kaiyuan Hu

PhD - McGill University

Chengming Hu

PhD - McGill University

Li Jiang Jiang

PhD - McGill University

Yili Jin

PhD - McGill University

Hong Kang

PhD - McGill University

Zonglun Li

PhD - McGill University

Yuyan Lin

Master's Research - McGill University

Junliang Luo

PhD - McGill University

Shaoxiang Qin

PhD - McGill University

Alejandro Salinas-Medina

PhD - McGill University

Zipeng Sun

PhD - McGill University

Haolun Wu

PhD - McGill University

Co-supervisor :

Fernando Diaz

Ye Yuan

PhD - McGill University

Importance-Aware Co-Teaching for Offline Model-Based Optimization

Weixu Zhang

PhD - McGill University

Blog Posts

Co-enseignement sensible à l’importance pour optimisation hors-ligne fondée sur un modèle

July 1, 2024

Ye Yuan

Can Chen

Zixuan Liu

Willie Neiswanger

Xue Liu

Read the article

September 20, 2023

Bidirectional Learning for Offline Model-based Optimization

Can Chen

Yingxue Zhang

Xue Liu

Mark Coates

Read the article

Publications

Policy Reuse for Communication Load Balancing in Unseen Traffic Scenarios

Yi Tian Xu

Jimmy Li

Di Wu

M. Jenkin

Seowoo Jang

With the continuous growth in communication network complexity and traffic volume, communication load balancing solutions are receiving incr… (see more)easing attention. Specifically, reinforcement learning (RL)-based methods have shown impressive performance compared with traditional rule-based methods. However, standard RL methods generally require an enormous amount of data to train, and generalize poorly to scenarios that are not encountered during training. We propose a policy reuse framework in which a policy selector chooses the most suitable pre-trained RL policy to execute based on the current traffic condition. Our method hinges on a policy bank composed of policies trained on a diverse set of traffic scenarios. When deploying to an unknown traffic scenario, we select a policy from the policy bank based on the similarity between the previous-day traffic of the current scenario and the traffic observed during training. Experiments demonstrate that this framework can outperform classical and adaptive rule-based methods by a large margin.

2023-06-01

ICC 2023 - IEEE International Conference on Communications (published)

Self-Supervised Transformer Architecture for Change Detection in Radio Access Networks

Igor Kozlov

Dmitriy Rivkin

Wei-Di Chang

Di Wu

Radio Access Networks (RANs) for telecommunications represent large agglomerations of interconnected hardware consisting of hundreds of thou… (see more)sands of transmitting devices (cells). Such networks undergo frequent and often heterogeneous changes caused by network operators, who are seeking to tune their system parameters for optimal performance. The effects of such changes are challenging to predict and will become even more so with the adoption of fifth-generation/sixth-generation (5G/6G) networks. Therefore, RAN monitoring is vital for network operators. We propose a self-supervised learning framework that leverages self-attention and self-distillation for this task. It works by detecting changes in Performance Measurement data, a collection of time-varying metrics which reflect a set of diverse measurements of the network performance at the cell level. Experimental results show that our approach outperforms the state of the art by 4% on a real-world based dataset consisting of about hundred thousands time series. It also has the merits of being scalable and generalizable. This allows it to provide deep insight into the specifics of mode of operation changes while relying minimally on expert knowledge.

2023-06-01

ICC 2023 - IEEE International Conference on Communications (published)

Reinforcement learning for communication load balancing: approaches and challenges

Di Wu

Jimmy Li

Amal Ferini

Yi Tian Xu

M. Jenkin

Seowoo Jang

The amount of cellular communication network traffic has increased dramatically in recent years, and this increase has led to a demand for e… (see more)nhanced network performance. Communication load balancing aims to balance the load across available network resources and thus improve the quality of service for network users. Most existing load balancing algorithms are manually designed and tuned rule-based methods where near-optimality is almost impossible to achieve. Furthermore, rule-based methods are difficult to adapt to quickly changing traffic patterns in real-world environments. Reinforcement learning (RL) algorithms, especially deep reinforcement learning algorithms, have achieved impressive successes in many application domains and offer the potential of good adaptabiity to dynamic changes in network load patterns. This survey presents a systematic overview of RL-based communication load-balancing methods and discusses related challenges and opportunities. We first provide an introduction to the load balancing problem and to RL from fundamental concepts to advanced models. Then, we review RL approaches that address emerging communication load balancing issues important to next generation networks, including 5G and beyond. Finally, we highlight important challenges, open issues, and future research directions for applying RL for communication load balancing.

2023-05-31

Frontiers of Computer Science (published)

Think Before You Act: Decision Transformers with Internal Working Memory

Jikun Kang

Romain Laroche

Xingdi Yuan

Adam Trischler

Jie Fu

Large language model (LLM)-based decision-making agents have shown the ability to generalize across multiple tasks. However, their performan… (see more)ce relies on massive data and compute. We argue that this inefficiency stems from the forgetting phenomenon, in which a model memorizes its behaviors in parameters throughout training. As a result, training on a new task may deteriorate the model's performance on previous tasks. In contrast to LLMs' implicit memory mechanism, the human brain utilizes distributed memory storage, which helps manage and organize multiple skills efficiently, mitigating the forgetting phenomenon. Thus inspired, we propose an internal working memory module to store, blend, and retrieve information for different downstream tasks. Evaluation results show that the proposed method improves training efficiency and generalization in both Atari games and meta-world object manipulation tasks. Moreover, we demonstrate that memory fine-tuning further enhances the adaptability of the proposed architecture.

2023-05-24

ArXiv (preprint)

Eliminating Space Scanning: Fast mmWave Beam Alignment with UWB Radios

Ju Wang

X. T. Chen

Due to their large bandwidth and impressive data speed, millimeter-wave (mmWave) radios are expected to play a key role in the 5G and beyond… (see more) (e.g., 6G) communication networks. Yet, to release mmWave’s true power, the highly directional mmWave beams need to be aligned perfectly. Most existing beam alignment methods adopt an exhaustive or semi-exhaustive space scanning, which introduces up to seconds of delays. To eliminate the need for complex space scanning, this article presents an Ultra-wideband (UWB)-assisted mmWave communication framework, which leverages the co-located UWB antennas to estimate the best angles for mmWave beam alignment. One major challenge of applying this idea in the real world is the barrier of limited antenna numbers. Commercial-Off-The-Shelf (COTS) devices are usually equipped with only a small number of UWB antennas, which are not enough for the existing algorithms to provide an accurate angle estimation. To solve this challenge, we design a novel Multi-Frequency MUltiple SIgnal Classification (MF-MUSIC) algorithm, which extends the classic MUltiple SIgnal Classification (MUSIC) algorithm to the frequency domain and overcomes the antenna limitation barrier in the spatial domain. Extensive real-world experiments and numerical simulations illustrate the advantage of the proposed MF-MUSIC algorithm. MF-MUSIC uses only three antennas to achieve an accurate angle estimation, which is a mere 0.15° (or a relative difference of 3.6%) different from the state-of-the-art 16-antenna-based angle estimation method.

2023-05-16

ACM Transactions on Sensor Networks (published)

Embracing Channel Estimation in Multi-Packet Reception of ZigBee

Zhe Wang

Linghe Kong

Guihai Chen

As a low-power and low-cost wireless protocol, the promising ZigBee has been widely used in sensor networks and cyber-physical systems. Sinc… (see more)e ZigBee based networks usually adopt tree or cluster topology, the convergecast scenarios are common in which multiple transmitters send packets to one receiver, leading to the severe collision problem. The conventional ZigBee adopts carrier sense multiple access with collisions avoidance to avoid collisions, which introduces additional time/energy overhead. The state-of-the-art methods resolve collisions instead of avoidance, in which mZig decomposes a collision by the collision itself and reZig decodes a collision by comparing with reference waveforms. However, mZig falls into high decoding errors only exploiting the signal amplitudes while reZig incurs high computational complexity for waveform comparison. In this paper, we propose CmZig to embrace channel estimation in multiple-packet reception (MPR) of ZigBee, which effectively improves MPR via lightweight computing used for channel estimation and collision decomposition. First, CmZig enables accurate collision decomposition with low computational complexity, which uses the estimated channel parameters modeling both signal amplitudes and phases. Second, CmZig adopts reference waveform comparison only for collisions without chip-level time offsets, instead of the complex machine learning based method. We implement CmZig on USRP-N210 and establish a six-node testbed. Results show that CmZig achieves a bit error rate in the order of

2023-05-01

IEEE Transactions on Mobile Computing (published)

Structure-aware protein self-supervised learning

Can Chen

Jingbo Zhou

F. Wang

Dejing Dou

2023-04-13

Bioinformatics (published)

Intent-aware Multi-source Contrastive Alignment for Tag-enhanced Recommendation

Haolun Wu

Yingxue Zhang

Chen Ma

Wei Guo

Ruiming Tang

Mark Coates

To offer accurate and diverse recommendation services, recent methods use auxiliary information to foster the learning process of user and i… (see more)tem representations. Many state-of-the-art (SOTA) methods fuse different sources of information (user, item, knowledge graph, tags, etc.) into a graph and use Graph Neural Networks (GNNs) to introduce the auxiliary information through the message passing paradigm. In this work, we seek an alternative framework that is light and effective through self-supervised learning across different sources of information, particularly for the commonly accessible item tag information. We use a self-supervision signal to pair users with the auxiliary information (tags) associated with the items they have interacted with before. To achieve the pairing, we create a proxy training task. For a given item, the model predicts which is the correct pairing between the representations obtained from the users that have interacted with this item and the tags assigned to it. This design provides an efficient solution, using the auxiliary information directly to enhance the quality of user and item embeddings. User behavior in recommendation systems is driven by the complex interactions of many factors behind the users’ decision-making processes. To make the pairing process more fine-grained and avoid embedding collapse, we propose a user intent-aware self-supervised pairing process where we split the user embeddings into multiple sub-embedding vectors. Each sub-embedding vector captures a specific user intent via self-supervised alignment with a particular cluster of tags. We integrate our designed framework with various recommendation models, demonstrating its flexibility and compatibility. Through comparison with numerous SOTA methods on seven real-world datasets, we show that our method can achieve better performance while requiring less training time. This indicates the potential of applying our approach on web-scale datasets.

2023-04-03

2023 IEEE 39th International Conference on Data Engineering (ICDE) (published)

Learning Multi-Objective Curricula for Robotic Policy Learning

Jikun Kang

Miao Liu

2023-03-06

Proceedings of The 6th Conference on Robot Learning (published)

proceedings.mlr.press

openreview.net

Ternary Quantization: A Survey

Danyang Liu

Inference time, model size, and accuracy are critical for deploying deep neural network models. Numerous research efforts have been made to … (see more)compress neural network models with faster inference and higher accuracy. Pruning and quantization are mainstream methods to this end. During model quantization, converting individual float values of layer weights to low-precision ones can substantially reduce the computational overhead and improve the inference speed. Many quantization methods have been studied, for example, vector quantization, low-bit quantization, and binary/ternary quantization. This survey focuses on ternary quantization. We review the evolution of ternary quantization and investigate the relationships among existing ternary quantization methods from the perspective of projection function and optimization methods.

2023-03-02

ArXiv (preprint)

Design and Implementation of Smooth Renewable Power in Cloud Data Centers

Xinxin Liu

Yu Hua

Ling Yang

Yuanyuan Sun

The renewable power has been widely used in modern cloud data centers, which also produce large electricity bills and the negative impacts o… (see more)n environments. However, frequent fluctuation and intermittency of renewable power often cause the challenges in terms of the stability of both electricity grid and data centers, as well as decreasing the utilization of renewable power. Existing schemes fail to alleviate the renewable power fluctuation, which is caused by the essential properties of renewable power. In order to address this problem, we propose an efficient and easy-to-use smooth renewable power-aware scheme, called Smoother, which consists of Flexible Smoothing (FS) and Active Delay (AD). First, in order to smooth the fluctuation of renewable power, FS carries out the optimized charge/discharge operation via computing the minimum variance of the renewable power that is supplied to data centers per interval. Second, AD improves the utilization of renewable power via actively adjusting the execution time of deferrable workloads. Extensive experimental results via examining the traces of real-world data centers demonstrate that Smoother significantly reduces the negative impact of renewable power fluctuations on data centers and improves the utilization of renewable power by 250.88 percent on average. We have released the source codes for public use.

2023-03-01

IEEE Transactions on Cloud Computing (published)

Learning From FM Communications: Toward Accurate, Efficient, All-Terrain Vehicle Localization

X. T. Chen

Qiao Xiang

Linghe Kong

Huisan Xu

Vehicle localization service is a fundamental component of intelligent transportation systems. The widely used satellite navigation systems … (see more)perform poorly in urban areas because the lines of sight to satellites are blocked by complex terrain characteristics, e.g., buildings, elevated streets and interchanges. In this paper, we design RadioLoc, a novel system achieving accurate, efficient, all-terrain vehicle localization with two key design points. First, RadioLoc harvests the frequency modulation (FM) signal, which has higher availability than satellite signal in complex terrains, as the signal source for localization. Second, RadioLoc integrates modern machine learning techniques into the processing of FM signals to efficiently learn the accurate vehicle localization in all-terrain environments. We validate the feasibility of FM-based vehicle localization and corresponding challenges and practical issues via field tests (e.g., signal distortion, signal inconsistency and limited in- vehicle radio bandwidth), and develop a series of advanced techniques in RadioLoc to address them, including adaptive batching, frequency sweeping, a novel multipath delay spread filter, a reconstructive PCA denoiser and a tailored FM feature extractor. We then develop a generic, modular localization module in RadioLoc, and design different learning-based 3D position identification algorithms for this module. We implement a prototype of RadioLoc and perform extensive field experiments to evaluate its efficiency and efficacy. Results show that (1) RadioLoc achieves a real-time localization latency of less than 100 milliseconds; (2) RadioLoc achieves a worst-case localization accuracy of 99.6% even in an underground parking lot, and (3) the horizontal error of RadioLoc is only one sixth of a dedicated GPS device even when the vehicle is moving at a high-speed (i.e., 80 km/h) in a complex highway scenario.

2023-02-01

IEEE/ACM Transactions on Networking (published)