Nous utilisons des témoins pour analyser le trafic et l’utilisation de notre site web, afin de personnaliser votre expérience. Vous pouvez désactiver ces technologies à tout moment, mais cela peut restreindre certaines fonctionnalités du site. Consultez notre Politique de protection de la vie privée pour en savoir plus.
Paramètre des cookies
Vous pouvez activer et désactiver les types de cookies que vous souhaitez accepter. Cependant certains choix que vous ferez pourraient affecter les services proposés sur nos sites (ex : suggestions, annonces personnalisées, etc.).
Cookies essentiels
Ces cookies sont nécessaires au fonctionnement du site et ne peuvent être désactivés. (Toujours actif)
Cookies analyse
Acceptez-vous l'utilisation de cookies pour mesurer l'audience de nos sites ?
Multimedia Player
Acceptez-vous l'utilisation de cookies pour afficher et vous permettre de regarder les contenus vidéo hébergés par nos partenaires (YouTube, etc.) ?
Publications
Combating False Negatives in Adversarial Imitation Learning
In adversarial imitation learning, a discriminator is trained to differentiate agent episodes from expert demonstrations representing the de… (voir plus)sired behavior. However, as the trained policy learns to be more successful, the negative examples (the ones produced by the agent) become increasingly similar to expert ones. Despite the fact that the task is successfully accomplished in some of the agent's trajectories, the discriminator is trained to output low values for them. We hypothesize that this inconsistent training signal for the discriminator can impede its learning, and consequently leads to worse overall performance of the agent. We show experimental evidence for this hypothesis and that the ‘False Negatives’ (i.e. successful agent episodes) significantly hinder adversarial imitation learning, which is the first contribution of this paper. Then, we propose a method to alleviate the impact of false negatives and test it on the BabyAI environment. This method consistently improves sample efficiency over the baselines by at least an order of magnitude.
2021-07-18
2021 International Joint Conference on Neural Networks (IJCNN) (publié)
This paper introduces a new framework for reducing mode collapse in Generative adversarial networks (GANs). The problem occurs when the gene… (voir plus)rator learns to map several various input values (z) to the same output value, which makes the generator fail to capture all modes of the true data distribution. As a result, the diversity of synthetically produced data is lower than that of the real data. To address this problem, we propose a new and simple framework for training GANs based on the concept of virtual mapping. Our framework integrates two processes into GANs: merge and split. The merge process merges multiple data points (samples) into one before training the discriminator. In this way, the generator would be trained to capture the merged-data distribution rather than the (unmerged) data distribution. After the training, the split process is applied to the generator's output in order to split its contents and produce diverse modes. The proposed framework increases the chance of capturing diverse modes through enabling an indirect or virtual mapping between an input z value and multiple data points. This, in turn, enhances the chance of generating more diverse modes. Our results show the effectiveness of our framework compared to the existing approaches in terms of reducing the mode collapse problem.
2021-07-18
IEEE International Joint Conference on Neural Network (publié)
The third edition of the International Classification of Sleep Disorders (ICSD‐3) is the authoritative clinical text for the diagnosis of … (voir plus)sleep disorders. An important issue of sleep nosology is to better understand the relationship between symptoms found in conventional diagnostic manuals and to compare classifications. Nevertheless, to our knowledge, there is no specific exhaustive work on the general structure of the networks of symptoms of sleep disorders as described in diagnostic manuals. The general aim of the present study was to use symptom network analysis to explore the diagnostic criteria in the ICSD‐3 manual. The ICSD‐3 diagnostic criteria related to clinical manifestations were systematically identified, and the units of analysis (symptoms) were labelled from these clinical manifestation diagnostic criteria using three rules (“Conservation”, “Splitting”, “Lumping”). A total of 37 of the 43 main sleep disorders with 160 units of analysis from 114 clinical manifestations in the ICSD‐3 were analysed. A symptom network representing all individual ICSD‐3 criteria and connections between them was constructed graphically (network estimation), quantified with classical metrics (network inference with global and local measures) and tested for robustness. The global measure of the sleep symptoms network shows that it can be considered as a small world, suggesting a strong interconnection between symptoms in the ICSD‐3. Local measures show the central role of three kinds of bridge sleep symptoms: daytime sleepiness, insomnia, and behaviour during sleep symptoms. Such a symptom network analysis of the ICSD‐3 structure could provide a framework for better systematising and organising symptomatology in sleep medicine.
When the question of who should get access to a communal resource first is uncertain, people often negotiate via nonverbal communication to … (voir plus)resolve the conflict. What should a robot be programmed to do when such conflicts arise in Human-Robot Interaction? The answer to this question varies depending on the context of the situation. Learning from how humans use hesitation gestures to negotiate a solution in such conflict situations, we present a human-inspired design of nonverbal hesitation gestures that can be used for Human-Robot Negotiation. We extracted characteristic features of such negotiative hesitations humans use, and subsequently designed a trajectory generator (Negotiative Hesitation Generator) that can re-create the features in robot responses to conflicts. Our human-subjects experiment demonstrates the efficacy of the designed robot behaviour against non-negotiative stopping behaviour of a robot. With positive results from our human-robot interaction experiment, we provide a validated trajectory generator with which one can explore the dynamics of human-robot nonverbal negotiation of resource conflicts.
2021-07-11
ACM Transactions on Human-Robot Interaction (publié)
Reasoning in a temporal knowledge graph (TKG) is a critical task for information retrieval and semantic search. It is particularly challengi… (voir plus)ng when the TKG is updated frequently. The model has to adapt to changes in the TKG for efficient training and inference while preserving its performance on historical knowledge. Recent work approaches TKG completion (TKGC) by augmenting the encoder-decoder framework with a time-aware encoding function. However, naively fine-tuning the model at every time step using these methods does not address the problems of 1) catastrophic forgetting, 2) the model's inability to identify the change of facts (e.g., the change of the political affiliation and end of a marriage), and 3) the lack of training efficiency. To address these challenges, we present the Time-aware Incremental Embedding (TIE) framework, which combines TKG representation learning, experience replay, and temporal regularization. We introduce a set of metrics that characterizes the intransigence of the model and propose a constraint that associates the deleted facts with negative labels. Experimental results on Wikidata12k and YAGO11k datasets demonstrate that the proposed TIE framework reduces training time by about ten times and improves on the proposed metrics compared to vanilla full-batch training. It comes without a significant loss in performance for any traditional measures. Extensive ablation studies reveal performance trade-offs among different evaluation metrics, which is essential for decision-making around real-world TKG applications.
2021-07-11
Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (publié)
Neurons are very complicated computational devices, incorporating numerous non-linear processes, particularly in their dendrites. Biophysica… (voir plus)l models capture these processes directly by explicitly modelling physiological variables, such as ion channels, current flow, membrane capacitance, etc. However, another option for capturing the complexities of real neural computation is to use cascade models, which treat individual neurons as a cascade of linear and non-linear operations, akin to a multi-layer artificial neural network. Recent research has shown that cascade models can capture single-cell computation well, but there are still a number of sub-cellular, regenerative dendritic phenomena that they cannot capture, such as the interaction between sodium, calcium, and NMDA spikes in different compartments. Here, we propose that it is possible to capture these additional phenomena using parallel, recurrent cascade models, wherein an individual neuron is modelled as a cascade of parallel linear and non-linear operations that can be connected recurrently, akin to a multi-layer, recurrent, artificial neural network. Given their tractable mathematical structure, we show that neuron models expressed in terms of parallel recurrent cascades can themselves be integrated into multi-layered artificial neural networks and trained to perform complex tasks. We go on to discuss potential implications and uses of these models for artificial intelligence. Overall, we argue that parallel, recurrent cascade models provide an important, unifying tool for capturing single-cell computation and exploring the algorithmic implications of physiological phenomena.
Neurons are very complicated computational devices, incorporating numerous non-linear processes, particularly in their dendrites. Biophysica… (voir plus)l models capture these processes directly by explicitly modelling physiological variables, such as ion channels, current flow, membrane capacitance, etc. However, another option for capturing the complexities of real neural computation is to use cascade models, which treat individual neurons as a cascade of linear and non-linear operations, akin to a multi-layer artificial neural network. Recent research has shown that cascade models can capture single-cell computation well, but there are still a number of sub-cellular, regenerative dendritic phenomena that they cannot capture, such as the interaction between sodium, calcium, and NMDA spikes in different compartments. Here, we propose that it is possible to capture these additional phenomena using parallel, recurrent cascade models, wherein an individual neuron is modelled as a cascade of parallel linear and non-linear operations that can be connected recurrently, akin to a multi-layer, recurrent, artificial neural network. Given their tractable mathematical structure, we show that neuron models expressed in terms of parallel recurrent cascades can themselves be integrated into multi-layered artificial neural networks and trained to perform complex tasks. We go on to discuss potential implications and uses of these models for artificial intelligence. Overall, we argue that parallel, recurrent cascade models provide an important, unifying tool for capturing single-cell computation and exploring the algorithmic implications of physiological phenomena.
Current deep reinforcement learning (RL) algorithms are still highly task-specific and lack the ability to generalize to new environments. L… (voir plus)ifelong learning (LLL), however, aims at solving multiple tasks sequentially by efficiently transferring and using knowledge between tasks. Despite a surge of interest in lifelong RL in recent years, the lack of a realistic testbed makes robust evaluation of LLL algorithms difficult. Multi-agent RL (MARL), on the other hand, can be seen as a natural scenario for lifelong RL due to its inherent non-stationarity, since the agents' policies change over time. In this work, we introduce a multi-agent lifelong learning testbed that supports both zero-shot and few-shot settings. Our setup is based on Hanabi -- a partially-observable, fully cooperative multi-agent game that has been shown to be challenging for zero-shot coordination. Its large strategy space makes it a desirable environment for lifelong RL tasks. We evaluate several recent MARL methods, and benchmark state-of-the-art LLL algorithms in limited memory and computation regimes to shed light on their strengths and weaknesses. This continual learning paradigm also provides us with a pragmatic way of going beyond centralized training which is the most commonly used training protocol in MARL. We empirically show that the agents trained in our setup are able to coordinate well with unseen agents, without any additional assumptions made by previous works. The code and all pre-trained models are available at https://github.com/chandar-lab/Lifelong-Hanabi.
2021-07-01
Proceedings of the 38th International Conference on Machine Learning (publié)
