Nous utilisons des témoins pour analyser le trafic et l’utilisation de notre site web, afin de personnaliser votre expérience. Vous pouvez désactiver ces technologies à tout moment, mais cela peut restreindre certaines fonctionnalités du site. Consultez notre Politique de protection de la vie privée pour en savoir plus.
Paramètre des cookies
Vous pouvez activer et désactiver les types de cookies que vous souhaitez accepter. Cependant certains choix que vous ferez pourraient affecter les services proposés sur nos sites (ex : suggestions, annonces personnalisées, etc.).
Cookies essentiels
Ces cookies sont nécessaires au fonctionnement du site et ne peuvent être désactivés. (Toujours actif)
Cookies analyse
Acceptez-vous l'utilisation de cookies pour mesurer l'audience de nos sites ?
Multimedia Player
Acceptez-vous l'utilisation de cookies pour afficher et vous permettre de regarder les contenus vidéo hébergés par nos partenaires (YouTube, etc.) ?
Publications
On Bonus Based Exploration Methods In The Arcade Learning Environment
Research on exploration in reinforcement learning, as applied to Atari 2600 game-playing, has emphasized tackling difficult exploration prob… (voir plus)lems such as Montezuma's Revenge (Bellemare et al., 2016). Recently, bonus-based exploration methods, which explore by augmenting the environment reward, have reached above-human average performance on such domains. In this paper we reassess popular bonus-based exploration methods within a common evaluation framework. We combine Rainbow (Hessel et al., 2018) with different exploration bonuses and evaluate its performance on Montezuma's Revenge, Bellemare et al.'s set of hard of exploration games with sparse rewards, and the whole Atari 2600 suite. We find that while exploration bonuses lead to higher score on Montezuma's Revenge they do not provide meaningful gains over the simpler epsilon-greedy scheme. In fact, we find that methods that perform best on that game often underperform epsilon-greedy on easy exploration Atari 2600 games. We find that our conclusions remain valid even when hyperparameters are tuned for these easy-exploration games. Finally, we find that none of the methods surveyed benefit from additional training samples (1 billion frames, versus Rainbow's 200 million) on Bellemare et al.'s hard exploration games. Our results suggest that recent gains in Montezuma's Revenge may be better attributed to architecture change, rather than better exploration schemes; and that the real pace of progress in exploration research for Atari 2600 games may have been obfuscated by good results on a single domain.
The SARS-CoV-2 (Covid-19) pandemic has resulted in significant strain on health care and public health institutions around the world. Contac… (voir plus)t tracing is an essential tool for public health officials and local communities to change the course of the Covid-19 pandemic. Standard manual contact tracing of people infected with Covid-19, while the current gold standard, has significant challenges that limit the ability of public health authorities to minimize community infections. Personalized peer-to-peer contact tracing through the use of mobile applications has the potential to shift the paradigm of Covid-19 community spread. Although some countries have deployed centralized tracking systems through either GPS or Bluetooth, more privacy-protecting decentralized systems offer much of the same benefit without concentrating data in the hands of a state authority or in for-profit corporations. Additionally, machine learning methods can be used to circumvent some of the limitations of standard digital tracing by incorporating many clues (including medical conditions, self-reported symptoms, and numerous encounters with people at different risk levels, for different durations and distances) and their uncertainty into a more graded and precise estimation of infection and contagion risk. The estimated risk can be used to provide early risk awareness, personalized recommendations and relevant information to the user and connect them to health services. Finally, the non-identifying data about these risks can inform detailed epidemiological models trained jointly with the machine learning predictor, and these models can provide statistical evidence for the interaction and importance of different factors involved in the transmission of the disease. They can also be used to monitor, evaluate and optimize different health policy and confinement/deconfinement scenarios according to medical and economic productivity indicators. However, such a strategy based on mobile apps and machine learning should proactively mitigate potential ethical and privacy risks, which could have substantial impacts on society (not only impacts on health but also impacts such as stigmatization and abuse of personal data). Here, we present an overview of the rationale, design, ethical considerations and privacy strategy of ‘COVI,’ a Covid-19 public peer-to-peer contact tracing and risk awareness mobile application developed in Canada. Addendum 2020-07-14: The government of Canada has declined to endorse COVI and will be promoting a different app for decentralized contact tracing. In the interest of preventing fragmentation of the app landscape, COVI will therefore not be deployed to end users. We are currently still in the process of finalizing the project, and plan to release our code and models for academic consumption and to make them accessible to other States should they wish to deploy an app based on or inspired by said code and models. University of Ottawa, Mila, Université de Montréal, The Alan Turing Institute, University of Oxford, University of Pennsylvania, McGill University, Borden Ladner Gervais LLP, The Decision Lab, HEC Montréal, Max Planck Institute, Libéo, University of Toronto. Corresponding author general: richard.janda@mcgill.ca Corresponding author for public health: abhinav.sharma@mcgill.ca Corresponding author for privacy: ywyu@math.toronto.edu Corresponding author for machine learning: yoshua.bengio@mila.quebec Corresponding author for user perspective: brooke@thedecisionlab.com Corresponding author for technical implementation: jean-francois.rousseau@libeo.com 1 ar X iv :2 00 5. 08 50 2v 2 [ cs .C R ] 2 7 Ju l 2 02 0
Cross-layer communication over fading channels with adaptive decision feedback
In this paper, cross-layer design of transmitting data packets over AWGN fading channel with adaptive decision feedback is considered. The t… (voir plus)ransmitter decides the number of packets to transmit and the threshold of the decision feedback based on the queue length and the channel state. The transmit power is chosen such that the probability of error is below a pre-specified threshold. We model the system as a Markov decision process and use ideas from lattice theory to establish qualitative properties of optimal transmission strategies. In particular, we show that: (i) if the channel state remains the same and the number of packets in the queue increase, then the optimal policy either transmits more packets or uses a smaller decision feedback threshold or both; and (ii) if the number of packets in the queue remain the same and the channel quality deteriorates, then the optimal policy either transmits fewer packets or uses a larger threshold for the decision feedback or both. We also show under rate constraints that if the channel gains for all channel states are above a threshold, then the “or” in the above characterization can be replaced by “and”. Finally, we present a numerical example showing that adaptive decision feedback significantly improves the power-delay trade-off as compared with the case of no feedback.
Discovering causal relationships in data is a challenging task that involves solving a combinatorial problem for which the solution is not a… (voir plus)lways identifiable. A new line of work reformulates the combinatorial problem as a continuous constrained optimization one, enabling the use of different powerful optimization techniques. However, methods based on this idea do not yet make use of interventional data, which can significantly alleviate identifiability issues. In this work, we propose a neural network-based method for this task that can leverage interventional data. We illustrate the flexibility of the continuous-constrained framework by taking advantage of expressive neural architectures such as normalizing flows. We show that our approach compares favorably to the state of the art in a variety of settings, including perfect and imperfect interventions for which the targeted nodes may even be unknown.
17 The human brain differs from that of other primates, but the genetic basis of these differences 18 remains unclear. We investigated the e… (voir plus)volutionary pressures acting on almost all human 19 protein-coding genes ( N =11,667; 1:1 orthologs in primates) on the basis of their divergence 20 from those of early hominins, such as Neanderthals, and non-human primates. We confirm 21 that genes encoding brain-related proteins are among the most strongly conserved protein- 22 coding genes in the human genome. Combining our evolutionary pressure metrics for the 23 protein-coding genome with recent datasets, we found that this conservation applied to genes 24 functionally associated with the synapse and expressed in brain structures such as the 25 prefrontal cortex and the cerebellum. Conversely, several of the protein-coding genes that 26 diverge most in hominins relative to other primates are associated with brain-associated 27 diseases, such as micro/macrocephaly, dyslexia, and autism. We also showed that cerebellum 28 granule neurons express a set of divergent protein-coding genes that may have contributed to 29 the emergence of fine motor skills and social cognition in humans. This resource is available 30 from http://neanderthal.pasteur.fr and can be used to estimate evolutionary constraints acting 31 on a set of genes and to explore their relative contributions to human traits. 32
On Efficiency in Hierarchical Reinforcement Learning
Hierarchical Reinforcement Learning (HRL) approaches promise to provide more efficient solutions to sequential decision making problems, bo… (voir plus)th in terms of statistical as well as computational efficiency. While this has been demonstrated empirically over time in a variety of tasks, theoretical results quantifying the ben-efits of such methods are still few and far between. In this paper, we discuss the kind of structure in a Markov decision process which gives rise to efficient HRL methods. Specifically, we formalize the intuition that HRL can exploit well repeating "subMDPs", with similar reward and transition structure. We show that, under reasonable assumptions, a model-based Thompson sampling-style HRL algorithm that exploits this structure is statistically efficient, as established through a finite-time regret bound. We also establish conditions under which planning with structure-induced options is near-optimal and computationally efficient.
Electric Vehicles Equilibrium Model that Considers Queue Delay and Mixed Traffic
This study develops an equilibrium model for electric vehicles (EVs) that considers both queue delays in charging stations and flow dependen… (voir plus)t travel times. This is a user equilibrium model that accounts for travel, charging and queuing time in the path choice modelling of EVs and the complementary traffic. Waiting and service times in charging stations are represented by an m/m/k queuing system. The model considers multiple vehicle and driver classes, expressing different battery capacity, initial charge state and range anxiety level. Feasible paths are found for multiple classes given their limited travel range. A numerical application exemplifies the limitations of EVs assignment and their impact on flow distribution.
AN ENSEMBLE APPROACH FOR DETECTING MACHINE FAILURE FROM SOUND Technical
We develop an ensemble-based approach for our submission to the anomaly detection challenge at DCASE 2020. The main members of our ensemble … (voir plus)are auto-encoders (with reconstruction error as the signal), classifiers (with negative predictive confidence as the signal), mismatch of the time-shifted signal with its Fourier-phase-shifted version, and a Gaussian mixture model on a set of common short-term features extracted from the waveform. The scores are passed through an exponential non-linearity and weighted to provide the final score, where the weighting and scaling hyper-parameters are learned on the development set. Our ensemble improves over the baseline on the development set.
An Equivalence between Loss Functions and Non-Uniform Sampling in Experience Replay
Prioritized Experience Replay (PER) is a deep reinforcement learning technique in which agents learn from transitions sampled with non-unifo… (voir plus)rm probability proportionate to their temporal-difference error. We show that any loss function evaluated with non-uniformly sampled data can be transformed into another uniformly sampled loss function with the same expected gradient. Surprisingly, we find in some environments PER can be replaced entirely by this new loss function without impact to empirical performance. Furthermore, this relationship suggests a new branch of improvements to PER by correcting its uniformly sampled loss function equivalent. We demonstrate the effectiveness of our proposed modifications to PER and the equivalent loss function in several MuJoCo and Atari environments.
