Publications

Reinforcement learning for communication load balancing: approaches and challenges

Di Wu

Jimmy Li

Amal Ferini

Yi Tian Xu

M. Jenkin

Seowoo Jang

Xue (Steve) Liu

Gregory Dudek

The amount of cellular communication network traffic has increased dramatically in recent years, and this increase has led to a demand for e… (see more)nhanced network performance. Communication load balancing aims to balance the load across available network resources and thus improve the quality of service for network users. Most existing load balancing algorithms are manually designed and tuned rule-based methods where near-optimality is almost impossible to achieve. Furthermore, rule-based methods are difficult to adapt to quickly changing traffic patterns in real-world environments. Reinforcement learning (RL) algorithms, especially deep reinforcement learning algorithms, have achieved impressive successes in many application domains and offer the potential of good adaptabiity to dynamic changes in network load patterns. This survey presents a systematic overview of RL-based communication load-balancing methods and discusses related challenges and opportunities. We first provide an introduction to the load balancing problem and to RL from fundamental concepts to advanced models. Then, we review RL approaches that address emerging communication load balancing issues important to next generation networks, including 5G and beyond. Finally, we highlight important challenges, open issues, and future research directions for applying RL for communication load balancing.

2023-05-31

Frontiers of Computer Science (published)

doi.org

Responsible Design Patterns for Machine Learning Pipelines

Saud Hakem Al Harbi

Lionel Nganyewou Tidjon

Foutse Khomh

Integrating ethical practices into the AI development process for artificial intelligence (AI) is essential to ensure safe, fair, and respon… (see more)sible operation. AI ethics involves applying ethical principles to the entire life cycle of AI systems. This is essential to mitigate potential risks and harms associated with AI, such as algorithm biases. To achieve this goal, responsible design patterns (RDPs) are critical for Machine Learning (ML) pipelines to guarantee ethical and fair outcomes. In this paper, we propose a comprehensive framework incorporating RDPs into ML pipelines to mitigate risks and ensure the ethical development of AI systems. Our framework comprises new responsible AI design patterns for ML pipelines identified through a survey of AI ethics and data management experts and validated through real-world scenarios with expert feedback. The framework guides AI developers, data scientists, and policy-makers to implement ethical practices in AI development and deploy responsible AI systems in production.

2023-05-31

ArXiv (preprint)

doi.org

arxiv.org

Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior

Ayush K Chakravarthy

Trang M. Nguyen

Anirudh Goyal

Yoshua Bengio

Michael Curtis Mozer

2023-05-31

ArXiv (preprint)

doi.org

arxiv.org

Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior

Ayush K Chakravarthy

Trang M. Nguyen

Anirudh Goyal

Yoshua Bengio

Michael Curtis Mozer

The aim of object-centric vision is to construct an explicit representation of the objects in a scene. This representation is obtained via a… (see more) set of interchangeable modules called \emph{slots} or \emph{object files} that compete for local patches of an image. The competition has a weak inductive bias to preserve spatial continuity; consequently, one slot may claim patches scattered diffusely throughout the image. In contrast, the inductive bias of human vision is strong, to the degree that attention has classically been described with a spotlight metaphor. We incorporate a spatial-locality prior into state-of-the-art object-centric vision models and obtain significant improvements in segmenting objects in both synthetic and real-world datasets. Similar to human visual attention, the combination of image content and spatial constraints yield robust unsupervised object-centric learning, including less sensitivity to model hyperparameters.

2023-05-31

ArXiv (preprint)

doi.org

arxiv.org

A graphical user interface for calculating the arterial input function during dynamic positron emission tomography

Y. Daoud

Liam Carroll

Shirin A. Enger

Purpose. Dynamic positron emission tomography (dPET) requires the acquisition of the arterial input function (AIF), conventionally obtained … (see more)via invasive arterial blood sampling. To obtain the AIF non-invasively, our group developed and combined two novel solutions consisting of (1) a detector, placed on a patient’s wrist during the PET scans to measure the radiation leaving the wrist and (2) a Geant4-based Monte Carlo simulation software. The simulations require patient-specific wrist geometry. The aim of this study was to develop a graphical user interface (GUI) allowing the user to import 2D ultrasound scans of a patient’s wrist, and measure the wrist features needed to calculate the AIF. Methods. The GUI elements were implemented using Qt5 and VTK-8.2.0. The user imports a patient’s wrist ultrasound scans, measures the radial artery and veins’ surface and depth to model a wrist phantom, then specifies the radioactive source used during the dPET scan. The phantom, the source, and the number of decay events are imported into the Geant4-based Monte Carlo software to run a simulation. In this study, 100 million decays of 18F and 68Ga were simulated in a wrist phantom designed based on an ultrasound scan. The detector’s efficiency was calculated and the results were analyzed using a clinical data processing algorithm developed in a previous study. Results. The detector’s total efficiency decreased by 3.5% for 18F and by 51.7% for 68Ga when using a phantom based on ultrasound scans compared to a generic wrist phantom. Similarly, the data processing algorithm’s accuracy decreased when using the patient-specific phantom, giving errors greater than 1.0% for both radioisotopes. Conclusions. This toolkit enables the user to run Geant4-based Monte Carlo simulations for dPET detector development applications using a patient-specific wrist phantom. Leading to a more precise simulation of the developed detector during dPET and the calculation of a personalized AIF.

2023-05-30

Physics in Medicine & Biology (published)

doi.org

The neuroconnectionist research programme

Adrien C. Doerig

R. Sommers

Katja Seeliger

Blake Richards

J. Ismael

Grace W. Lindsay

Konrad Paul Kording

Talia Konkle

M. Gerven

Nikolaus Kriegeskorte

Tim Kietzmann

2023-05-30

Nature Reviews Neuroscience (published)

doi.org

arxiv.org

Attention Schema in Neural Agents

Dianbo Liu

Samuele Bolotta

Mike He Zhu

Yoshua Bengio

Guillaume Dumas

Attention has become a common ingredient in deep learning architectures. It adds a dynamical selection of information on top of the static s… (see more)election of information supported by weights. In the same way, we can imagine a higher-order informational filter built on top of attention: an Attention Schema (AS), namely, a descriptive and predictive model of attention. In cognitive neuroscience, Attention Schema Theory (AST) supports this idea of distinguishing attention from AS. A strong prediction of this theory is that an agent can use its own AS to also infer the states of other agents' attention and consequently enhance coordination with other agents. As such, multi-agent reinforcement learning would be an ideal setting to experimentally test the validity of AST. We explore different ways in which attention and AS interact with each other. Our preliminary results indicate that agents that implement the AS as a recurrent internal control achieve the best performance. In general, these exploratory experiments suggest that equipping artificial agents with a model of attention can enhance their social intelligence.

2023-05-27

ArXiv (preprint)

doi.org

arxiv.org

Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets

Dinghuai Zhang

Hanjun Dai

Nikolay Malkin

Aaron Courville

Yoshua Bengio

Ling Pan

Combinatorial optimization (CO) problems are often NP-hard and thus out of reach for exact algorithms, making them a tempting domain to appl… (see more)y machine learning methods. The highly structured constraints in these problems can hinder either optimization or sampling directly in the solution space. On the other hand, GFlowNets have recently emerged as a powerful machinery to efficiently sample from composite unnormalized densities sequentially and have the potential to amortize such solution-searching processes in CO, as well as generate diverse solution candidates. In this paper, we design Markov decision processes (MDPs) for different combinatorial problems and propose to train conditional GFlowNets to sample from the solution space. Efficient training techniques are also developed to benefit long-range credit assignment. Through extensive experiments on a variety of different CO tasks with synthetic and realistic data, we demonstrate that GFlowNet policies can efficiently find high-quality solutions. Our implementation is open-sourced at https://github.com/zdhNarsil/GFlowNet-CombOpt.

2023-05-26

ArXiv (preprint)

doi.org

arxiv.org

Motor cortex latent dynamics encode arm movement direction and urgency independently

Andrea Colins Rodriguez

Matt Perich

Lee Miller

Mark D. Humphries

2023-05-26

bioRxiv (preprint)

doi.org