Gregory Dudek

hanqing.zhao@mila.quebec

Publications

Learning Heuristics for Transit Network Design and Improvement with Deep Reinforcement Learning

Andrew Holliday

A. El-geneidy

2024-04-08

ArXiv (preprint)

Constrained Robotic Navigation on Preferred Terrains Using LLMs and Speech Instruction: Exploiting the Power of Adverbs

Faraz Lotfi

Farnoosh Faraji

Nikhil Kakodkar

Travis Manderson

David Meger

2024-04-02

ArXiv (preprint)

A Neural-Evolutionary Algorithm for Autonomous Transit Network Design

Andrew Holliday

2024-02-27

ArXiv (preprint)

A comparison of RL-based and PID controllers for 6-DOF swimming robots: hybrid underwater object tracking

Faraz Lotfi

Khalil Virji

Nicholas Dudek

2024-01-29

ArXiv (preprint)

Probabilistic Mobility Load Balancing for Multi-band 5G and Beyond Networks

Saria Al Laham

Di Wu

Ekram Hossain

2024-01-24

ArXiv (preprint)

PhotoBot: Reference-Guided Interactive Photography via Natural Language

Oliver Limoyo

Jimmy Li

Dmitriy Rivkin

Jonathan Kelly

We introduce PhotoBot, a framework for fully automated photo acquisition based on an interplay between high-level human language guidance an… (see more)d a robot photographer. We propose to communicate photography suggestions to the user via reference images that are selected from a curated gallery. We leverage a visual language model (VLM) and an object detector to characterize the reference images via textual descriptions and then use a large language model (LLM) to retrieve relevant reference images based on a user's language query through text-based reasoning. To correspond the reference image and the observed scene, we exploit pre-trained features from a vision transformer capable of capturing semantic similarity across marked appearance variations. Using these features, we compute pose adjustments for an RGB-D camera by solving a perspective-n-point (PnP) problem. We demonstrate our approach using a manipulator equipped with a wrist camera. Our user studies show that photos taken by PhotoBot are often more aesthetically pleasing than those taken by users themselves, as measured by human feedback. We also show that PhotoBot can generalize to other reference sources such as paintings.

2024-01-19

ArXiv (preprint)

Hallucination Detection and Hallucination Mitigation: An Investigation

Junliang Luo

Tianyu Li

Di Wu

M. Jenkin

Steve Liu

2024-01-16

ArXiv (preprint)

Device-Free Human State Estimation using UWB Multi-Static Radios

Saria Al Laham

Bobak H. Baghi

Pierre-Yves Lajoie

Amal Feriani

Sachini Herath

Steve Liu

We present a human state estimation framework that allows us to estimate the location, and even the activities, of people in an indoor envir… (see more)onment without the requirement that they carry a specific devices with them. To achieve this"device free"localization we use a small number of low-cost Ultra-Wide Band (UWB) sensors distributed across the environment of interest. To achieve high quality estimation from the UWB signals merely reflected of people in the environment, we exploit a deep network that can learn to make inferences. The hardware setup consists of commercial off-the-shelf (COTS) single antenna UWB modules for sensing, paired with Raspberry PI units for computational processing and data transfer. We make use of the channel impulse response (CIR) measurements from the UWB sensors to estimate the human state - comprised of location and activity - in a given area. Additionally, we can also estimate the number of humans that occupy this region of interest. In our approach, first, we pre-process the CIR data which involves meticulous aggregation of measurements and extraction of key statistics. Afterwards, we leverage a convolutional deep neural network to map the CIRs into precise location estimates with sub-30 cm accuracy. Similarly, we achieve accurate human activity recognition and occupancy counting results. We show that we can quickly fine-tune our model for new out-of-distribution users, a process that requires only a few minutes of data and a few epochs of training. Our results show that UWB is a promising solution for adaptable smart-home localization and activity recognition problems.

2023-12-26

ArXiv (preprint)

Anomaly Detection for Scalable Task Grouping in Reinforcement Learning-based RAN Optimization

Jimmy Li

Igor Kozlov

Di Wu

The use of learning-based methods for optimizing cellular radio access networks (RAN) has received increasing attention in recent years. Thi… (see more)s coincides with a rapid increase in the number of cell sites worldwide, driven largely by dramatic growth in cellular network traffic. Training and maintaining learned models that work well across a large number of cell sites has thus become a pertinent problem. This paper proposes a scalable framework for constructing a reinforcement learning policy bank that can perform RAN optimization across a large number of cell sites with varying traffic patterns. Central to our framework is a novel application of anomaly detection techniques to assess the compatibility between sites (tasks) and the policy bank. This allows our framework to intelligently identify when a policy can be reused for a task, and when a new policy needs to be trained and added to the policy bank. Our results show that our approach to compatibility assessment leads to an efficient use of computational resources, by allowing us to construct a performant policy bank without exhaustively training on all tasks, which makes it applicable under real-world constraints.

2023-12-06

ArXiv (preprint)

AdaTeacher: Adaptive Multi-Teacher Weighting for Communication Load Forecasting

Chengming Hu

Ju Wang

Di Wu

Yan Xin

Charlie Zhang

To deal with notorious delays in communication systems, it is crucial to forecast key system characteristics, such as the communication load… (see more). Most existing studies aggregate data from multiple edge nodes for improving the forecasting accuracy. However, the bandwidth cost of such data aggregation could be unacceptably high from the perspective of system operators. To achieve both the high forecasting accuracy and bandwidth efficiency, this paper proposes an Adaptive Multi-Teacher Weighting in Teacher-Student Learning approach, namely AdaTeacher, for communication load forecasting of multiple edge nodes. Each edge node trains a local model on its own data. A target node collects multiple models from its neighbor nodes and treats these models as teachers. Then, the target node trains a student model from teachers via Teacher-Student (T-S) learning. Unlike most existing T-S learning approaches that treat teachers evenly, resulting in a limited performance, AdaTeacher introduces a bilevel optimization algorithm to dynamically learn an importance weight for each teacher toward a more effective and accurate T-S learning process. Compared to the state-of-the-art methods, Ada Teacher not only reduces the bandwidth cost by 53.85%, but also improves the load forecasting accuracy by 21.56% and 24.24% on two real-world datasets.

2023-12-04

Global Communications Conference (published)

Energy Saving in Cellular Wireless Networks via Transfer Deep Reinforcement Learning

Di Wu

Yi Tian Xu

M. Jenkin

Seowoo Jang

Ekram Hossain

With the increasing use of data-intensive mobile applications and the number of mobile users, the demand for wireless data services has been… (see more) increasing exponentially in recent years. In order to address this demand, a large number of new cellular base stations are being deployed around the world, leading to a significant increase in energy consumption and greenhouse gas emission. Consequently, energy consumption has emerged as a key concern in the fifth-generation (5G) network era and beyond. Reinforcement learning (RL), which aims to learn a control policy via interacting with the environment, has been shown to be effective in addressing network optimization problems. However, for reinforcement learning, especially deep reinforcement learning, a large number of interactions with the environment are required. This often limits its applicability in the real world. In this work, to better deal with dynamic traffic scenarios and improve real-world applicability, we propose a transfer deep reinforcement learning framework for energy optimization in cellular communication networks. Specifically, we first pre-train a set of RL-based energy-saving policies on source base stations and then transfer the most suitable policy to the given target base station in an unsupervised learning manner. Experimental results demonstrate that base station energy consumption can be reduced significantly using this approach.

2023-12-04

Global Communications Conference (published)

Learning to Adapt: Communication Load Balancing via Adaptive Deep Reinforcement Learning

Di Wu

Yi Tian Xu

Jimmy Li

M. Jenkin

Ekram Hossain

Seowoo Jang

Yan Xin

Charlie Zhang

The association of mobile devices with network resources (e.g., base stations, frequency bands/channels), known as load balancing, is critic… (see more)al to reduce communication traffic congestion and network performance. Reinforcement learning (RL) has shown to be effective for communication load balancing and achieves better performance than currently used rule-based methods, especially when the traffic load changes quickly. However, RL-based methods usually need to interact with the environment for a large number of time steps to learn an effective policy and can be difficult to tune. In this work, we aim to improve the data efficiency of RL-based solutions to make them more suitable and applicable for real-world applications. Specifically, we propose a simple, yet efficient and effective deep RL-based wireless network load balancing framework. In this solution, a set of good initialization values for control actions are selected with some cost-efficient approach to center the training of the RL agent. Then, a deep RL-based agent is trained to find offsets from the initialization values that optimize the load balancing problem. Experimental evaluation on a set of dynamic traffic scenarios demonstrates the effectiveness and efficiency of the proposed method.

2023-12-04

GLOBECOM 2023 - 2023 IEEE Global Communications Conference (published)