Publications

Dissecting Deep RL with High Update Ratios: Combatting Value Overestimation and Divergence
Marcel Hussing
Claas Voelcker
Igor Gilitschenski
Amir-massoud Farahmand
Eric R. Eaton
We show that deep reinforcement learning can maintain its ability to learn without resetting network parameters in settings where the number… (see more) of gradient updates greatly exceeds the number of environment samples. Under such large update-to-data ratios, a recent study by Nikishin et al. (2022) suggested the emergence of a primacy bias , in which agents overfit early interactions and downplay later experience, impairing their ability to learn. In this work, we dissect the phenomena underlying the primacy bias. We inspect the early stages of training that ought to cause the failure to learn and find that a fundamental challenge is a long-standing acquaintance: value overestimation. Overinflated Q-values are found not only on out-of-distribution but also in-distribution data and can be traced to unseen action prediction propelled by optimizer momentum. We employ a simple unit-ball normalization that enables learning under large update ratios, show its efficacy on the widely used dm_control suite, and obtain strong performance on the challenging dog tasks, competitive with model-based approaches. Our results question, in parts, the prior explanation for sub-optimal learning due to overfitting on early data.
2024-02-29
arXiv (published)
doi.org
An Exact Method for (Constrained) Assortment Optimization Problems with Product Costs
Markus Leitner
Andrea Lodi
Roberto Roberti
Claudio Sole
2024-02-29
INFORMS Journal on Computing (published)
doi.org
arxiv.org
FedSwarm: An Adaptive Federated Learning Framework for Scalable AIoT
Haizhou Du
Chengdong Ni
Chaoqian Cheng
Qiao Xiang
X. T. Chen
Xue Liu
Federated learning (FL) is a key solution for datadriven the Artificial Intelligence of Things (AIoT). Although much progress has been made,… (see more) scalability remains a core challenge for real-world FL deployments. Existing solutions either suffer from accuracy loss or do not fully address the connectivity dynamicity of FL systems. In this article, we tackle the scalability issue with a novel, adaptive FL framework called FedSwarm, which improves system scalability for AIoT by deploying multiple collaborative edge servers. FedSwarm has two novel features: 1) adaptiveness on the number of local updates and 2) dynamicity of the synchronization between edge devices and edge servers. We formulate FedSwarm as a local update adaptation and perdevice dynamic server selection problem and prove FedSwarm‘s convergence bound. We further design a control mechanism consisting of a learning-based algorithm for collaboratively providing local update adaptation on the servers’ side and a bonus-based strategy for spurring dynamic per-device server selection on the devices’ side. Our extensive evaluation shows that FedSwarm significantly outperforms other studies with better scalability, lower energy consumption, and higher model accuracy.
2024-02-29
IEEE Internet of Things Journal (published)
doi.org
Gaussian-process-based Bayesian optimization for neurostimulation interventions in rats
Leo Choiniere
Rose Guay-Hottin
Rémi Picard
Guillaume Lajoie
Marco Bonizzato
Numa Dancause
2024-02-29
STAR Protocols (published)
doi.org
Generalization of deep learning models for hepatic steatosis grading using B-mode ultrasound images
Pedro Vianna
Yue Qi
Michaël Chassé
Guy Wolf
Eugene Belilovsky
An Tang
Guy Cloutier
Grayscale ultrasound remains a key modality for screening of hepatic steatosis due to its non-invasiveness and availability. While neural ne… (see more)tworks have shown promise in this field, their main drawback lies in their inability to generalize to diverse real-world settings. Variations in equipment, acquisition parameters, or population significantly affect model performance. Test-time adaptation, an unsupervised domain adaptation technique, overcomes these limitations by adjusting trained models during inference. Our retrospective study used two datasets collected in separate populations, with different scanners and protocols. We propose an adaptation method, using test-time batch normalization to selectively adjust BatchNorm layers based on test data for predicting steatosis grades. Comparing the non-adapted and adapted models, the mean absolute error (± standard deviation) in grading four severities of steatosis decreased from 0.92 ± 0.21 to 0.64 ± 0.22 . Specifically, for detection of steatosis the area under the curve increased from 0.76 ± 0.05 to 0.95 ± 0.02 when using the adapted model. Adapted models show promising results in improving performance compared to base models when testing data differ significantly from training data. Results suggest that the proposed method effectively addresses domain shift in diagnosing fatty liver using ultrasound images, reducing risks associated with deploying trained models.
2024-02-29
The Journal of the Acoustical Society of America (published)
doi.org
Implications of conscious AI in primary healthcare
Dorsai Ranjbari
Samira Abbasgholizadeh Rahimi
The conversation about consciousness of artificial intelligence (AI) is an ongoing topic since 1950s. Despite the numerous applications of A… (see more)I identified in healthcare and primary healthcare, little is known about how a conscious AI would reshape its use in this domain. While there is a wide range of ideas as to whether AI can or cannot possess consciousness, a prevailing theme in all arguments is uncertainty. Given this uncertainty and the high stakes associated with the use of AI in primary healthcare, it is imperative to be prepared for all scenarios including conscious AI systems being used for medical diagnosis, shared decision-making and resource management in the future. This commentary serves as an overview of some of the pertinent evidence supporting the use of AI in primary healthcare and proposes ideas as to how consciousnesses of AI can support or further complicate these applications. Given the scarcity of evidence on the association between consciousness of AI and its current state of use in primary healthcare, our commentary identifies some directions for future research in this area including assessing patients’, healthcare workers’ and policy-makers’ attitudes towards consciousness of AI systems in primary healthcare settings.
2024-02-29
Family Medicine and Community Health (published)
doi.org
Iterative Graph Self-Distillation
Hanlin Zhang
Shuai Lin
Weiyang Liu
Pan Zhou
Jian Tang
Xiaodan Liang
Eric P. Xing
Recently, there has been increasing interest in the challenge of how to discriminatively vectorize graphs. To address this, we propose a met… (see more)hod called Iterative Graph Self-Distillation (IGSD) which learns graph-level representation in an unsupervised manner through instance discrimination using a self-supervised contrastive learning approach. IGSD involves a teacher-student distillation process that uses graph diffusion augmentations and constructs the teacher model using an exponential moving average of the student model. The intuition behind IGSD is to predict the teacher network representation of the graph pairs under different augmented views. As a natural extension, we also apply IGSD to semi-supervised scenarios by jointly regularizing the network with both supervised and self-supervised contrastive loss. Finally, we show that fine-tuning the IGSD-trained models with self-training can further improve graph representation learning. Empirically, we achieve significant and consistent performance gain on various graph datasets in both unsupervised and semi-supervised settings, which well validates the superiority of IGSD.
2024-02-29
IEEE Transactions on Knowledge and Data Engineering (published)
doi.org
openreview.net
Joint Multimodal Transformer for Dimensional Emotional Recognition in the Wild
Paul Waligora
Muhammad Osama Zeeshan
Muhammad Haseeb Aslam
Soufiane Belharbi
Alessandro Lameiras Koerich
Marco Pedersoli
Simon Bacon
Eric Granger
Audiovisual emotion recognition (ER) in videos has immense potential over unimodal performance. It effectively leverages the inter-and intra… (see more)-modal dependencies between visual and auditory modalities. This work proposes a novel audio-visual emotion recognition system utilizing a joint multimodal transformer architecture with key-based cross-attention. This framework aims to exploit the complementary nature of audio and visual cues (facial expressions and vocal patterns) in videos, leading to superior performance compared to solely relying on a single modality. The proposed model leverages separate backbones for capturing intra-modal temporal dependencies within each modality (audio and visual). Subse-quently, a joint multimodal transformer architecture integrates the individual modality embeddings, enabling the model to effectively capture inter-modal (between audio and visual) and intra-modal (within each modality) relationships. Extensive evaluations on the challenging Affwild2 dataset demonstrate that the proposed model significantly outperforms baseline and state-of-the-art methods in ER tasks.
2024-02-29
arXiv (published)
doi.org
Neural network prediction of the effect of thermomechanical controlled processing on mechanical properties
Sushant Sinha
Denzel Guye
Xiaoping Ma
Kashif Rehman
Stephen Yue
Narges Armanfard
2024-02-29
Machine Learning with Applications (published)
doi.org
Novel community data in ecology-properties and prospects.
Florian Hartig
Nerea Abrego
Alex Bush
Jonathan M. Chase
G. Guillera‐Arroita
M. Leibold
Otso T. Ovaskainen
Loïc Pellissier
Maximilian Pichler
Giovanni Poggiato
Laura J. Pollock
Sara Si-moussi
Wilfried Thuiller
Duarte S Viana
D. Warton
Damaris Zurell
Douglas W. Yu
2024-02-29
Trends in Ecology & Evolution (published)
doi.org
arxiv.org
NOx emissions prediction for MSWI process based on dynamic modular neural network
Haoshan Duan
Xi Meng
Jian Tang
JunFei Qiao
2024-02-29
Expert Systems with Applications (published)
doi.org
Sources of Richness and Ineffability for Phenomenally Conscious States
Xu Ji
Conscious states (states that there is something it is like to be in) seem both rich or full of detail, and ineffable or hard to fully descr… (see more)ibe or recall. The problem of ineffability, in particular, is a longstanding issue in philosophy that partly motivates the explanatory gap: the belief that consciousness cannot be reduced to underlying physical processes. Here, we provide an information theoretic dynamical systems perspective on the richness and ineffability of consciousness. In our framework, the richness of conscious experience corresponds to the amount of information in a conscious state and ineffability corresponds to the amount of information lost at different stages of processing. We describe how attractor dynamics in working memory would induce impoverished recollections of our original experiences, how the discrete symbolic nature of language is insufficient for describing the rich and high-dimensional structure of experiences, and how similarity in the cognitive function of two individuals relates to improved communicability of their experiences to each other. While our model may not settle all questions relating to the explanatory gap, it makes progress toward a fully physicalist explanation of the richness and ineffability of conscious experience: two important aspects that seem to be part of what makes qualitative character so puzzling.
2024-02-29
Neuroscience of Consciousness (published)
doi.org
arxiv.org