Publications

Value Preserving State-Action Abstractions
David Abel
Nathan Umbanhowar
Khimya Khetarpal
Dilip Arumugam
Doina Precup
Michael L. Littman
Abstraction can improve the sample efficiency of reinforcement learning. However, the process of abstraction inherently discards information… (voir plus), potentially compromising an agent’s ability to represent high-value policies. To mitigate this, we here introduce combinations of state abstractions and options that are guaranteed to preserve the representation of near-optimal policies. We first define φ-relative options, a general formalism for analyzing the value loss of options paired with a state abstraction, and present necessary and sufficient conditions for φ-relative options to preserve near-optimal behavior in any finite Markov Decision Process. We further show that, under appropriate assumptions, φ-relative options can be composed to induce hierarchical abstractions that are also guaranteed to represent high-value policies.ion can improve the sample efficiency of reinforcement learning. However, the process of abstraction inherently discards information, potentially compromising an agent’s ability to represent high-value policies. To mitigate this, we here introduce combinations of state abstractions and options that are guaranteed to preserve the representation of near-optimal policies. We first define φ-relative options, a general formalism for analyzing the value loss of options paired with a state abstraction, and present necessary and sufficient conditions for φ-relative options to preserve near-optimal behavior in any finite Markov Decision Process. We further show that, under appropriate assumptions, φ-relative options can be composed to induce hierarchical abstractions that are also guaranteed to represent high-value policies.
2020-06-03
International Conference on Artificial Intelligence and Statistics (published)
dblp.uni-trier.de
Value Preserving State-Action Abstractions
David Abel
Nathan Umbanhowar
Khimya Khetarpal
Dilip Arumugam
Doina Precup
Michael L. Littman
Abstraction can improve the sample efficiency of reinforcement learning. However, the process of abstraction inherently discards information… (voir plus), potentially compromising an agent’s ability to represent high-value policies. To mitigate this, we here introduce combinations of state abstractions and options that are guaranteed to preserve the representation of near-optimal policies. We first define φ-relative options, a general formalism for analyzing the value loss of options paired with a state abstraction, and present necessary and sufficient conditions for φ-relative options to preserve near-optimal behavior in any finite Markov Decision Process. We further show that, under appropriate assumptions, φ-relative options can be composed to induce hierarchical abstractions that are also guaranteed to represent high-value policies.ion can improve the sample efficiency of reinforcement learning. However, the process of abstraction inherently discards information, potentially compromising an agent’s ability to represent high-value policies. To mitigate this, we here introduce combinations of state abstractions and options that are guaranteed to preserve the representation of near-optimal policies. We first define φ-relative options, a general formalism for analyzing the value loss of options paired with a state abstraction, and present necessary and sufficient conditions for φ-relative options to preserve near-optimal behavior in any finite Markov Decision Process. We further show that, under appropriate assumptions, φ-relative options can be composed to induce hierarchical abstractions that are also guaranteed to represent high-value policies.
2020-06-03
International Conference on Artificial Intelligence and Statistics (publié)
proceedings.mlr.press
Restless bandits: indexability and computation of Whittle index
Nima Akbarzadeh
Aditya Mahajan
Restless bandits are a class of sequential resource allocation problems concerned with allocating one or more resources among several altern… (voir plus)ative processes where the evolution of the process depends on the resource allocated to them. Such models capture the fundamental trade-offs between exploration and exploitation. In 1988, Whittle developed an index heuristic for restless bandit problems which has emerged as a popular solution approach due to its simplicity and strong empirical performance. The Whittle index heuristic is applicable if the model satisfies a technical condition known as indexability. In this paper, we present two general sufficient conditions for indexability and identify simpler to verify refinements of these conditions. We then present a general algorithm to compute Whittle index for indexable restless bandits. Finally, we present a detailed numerical study which affirms the strong performance of the Whittle index heuristic.
2020-06-01
arXiv.org (prépublication)
dblp.uni-trier.de
GIANT: Scalable Creation of a Web-scale Ontology
Bang Liu
Weidong Guo
Di Niu
Jinwen Luo
Chaoyue Wang
Zhen Wen
Yu Xu
2020-05-31
Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (publié)
doi.org
arxiv.org
Current works and future directions on application of machine learning in primary care
Samira Abbasgholizadeh-Rahimi
Vera Granikov
Pierre Pluye
In this short paper, we explained current machine learning works in primary care based on a scoping review that we performed. The performed … (voir plus)review was in line with the methodological framework proposed by Colquhoun and colleagues. Lastly, we discussed our observations and gave important directions to the future studies in this fast-growing area.
2020-05-27
International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (publié)
doi.org
Failure to follow medication changes made at hospital discharge is associated with adverse events in 30 days
Daniala L Weir
Aude Motulsky
Michal Abrahamowicz
Todd C. Lee
Steven Morgan
David Buckeridge
Robyn Tamblyn
2020-05-20
Health Services Research (publié)
doi.org
Evaluating White Matter Lesion Segmentations with Refined Sørensen-Dice Analysis
Aaron Carass
Snehashis Roy
Adrian Gherman
Jacob C. Reinhold
Andrew Jesson
Tal Arbel
Oskar Maier
Heinz Handels
Mohsen Ghafoorian
Bram Platel
Ariel Birenbaum
Hayit Greenspan
Dzung L. Pham
Ciprian M. Crainiceanu
Peter A. Calabresi
Jerry L. Prince
William R. Gray Roncal
Russell T. Shinohara
Ipek Oguz
2020-05-19
Scientific Reports (publié)
doi.org
An Analysis of the Adaptation Speed of Causal Models
Rémi LE PRIOL
Reza Babanezhad Harikandeh
Yoshua Bengio
Simon Lacoste-Julien
We consider the problem of discovering the causal process that generated a collection of datasets. We assume that all these datasets were ge… (voir plus)nerated by unknown sparse interventions on a structural causal model (SCM)
2020-05-18
ArXiv (preprint)
arxiv.org
arxiv.org
COVI White Paper
Hannah Alsdurf
Yoshua Bengio
Tristan Deleu
Prateek Gupta
Daphne Ippolito
Richard Janda
Max Jarvie
Tyler J. Kolody
Sekoul Krastev
Tegan Maharaj
Robert Obryk
Dan Pilat
Valerie Pisano
Benjamin Prud'homme
Meng Qu
Nasim Rahaman
Irina Rish
Jean-franois Rousseau
abhinav sharma
Brooke Struck … (voir 3 de plus)
Jian Tang
Martin Weiss
Yun William Yu
2020-05-18
ArXiv (prépublication)
arxiv.org
arxiv.org
Story Forest
Bang Liu
Fred X. Han
Di Niu
Linglong Kong
Kunfeng Lai
Yu Xu
Extracting events accurately from vast news corpora and organize events logically is critical for news apps and search engines, which aim to… (voir plus) organize news information collected from the Internet and present it to users in the most sensible forms. Intuitively speaking, an event is a group of news documents that report the same news incident possibly in different ways. In this article, we describe our experience of implementing a news content organization system at Tencent to discover events from vast streams of breaking news and to evolve news story structures in an online fashion. Our real-world system faces unique challenges in contrast to previous studies on topic detection and tracking (TDT) and event timeline or graph generation, in that we (1) need to accurately and quickly extract distinguishable events from massive streams of long text documents, and (2) must develop the structures of event stories in an online manner, in order to guarantee a consistent user viewing experience. In solving these challenges, we propose Story Forest, a set of online schemes that automatically clusters streaming documents into events, while connecting related events in growing trees to tell evolving stories. A core novelty of our Story Forest system is EventX, a semi-supervised scheme to extract events from massive Internet news corpora. EventX relies on a two-layered, graph-based clustering procedure to group documents into fine-grained events. We conducted extensive evaluations based on (1) 60 GB of real-world Chinese news data, (2) a large Chinese Internet news dataset that contains 11,748 news articles with truth event labels, and (3) the 20 News Groups English dataset, through detailed pilot user experience studies. The results demonstrate the superior capabilities of Story Forest to accurately identify events and organize news text into a logical structure that is appealing to human readers.
2020-05-13
ACM Transactions on Knowledge Discovery from Data (publié)
doi.org
Leveraging exploration in off-policy algorithms via normalizing flows
Bogdan Mazoure
Thang Doan
Audrey Durand
Joelle Pineau
(Rex) Devon Hjelm
Exploration is a crucial component for discovering approximately optimal policies in most high-dimensional reinforcement learning (RL) setti… (voir plus)ngs with sparse rewards. Approaches such as neural density models and continuous exploration (e.g., Go-Explore) have been instrumental in recent advances. Soft actor-critic (SAC) is a method for improving exploration that aims to combine off-policy updates while maximizing the policy entropy. We extend SAC to a richer class of probability distributions through normalizing flows, which we show improves performance in exploration, sample complexity, and convergence. Finally, we show that not only the normalizing flow policy outperforms SAC on MuJoCo domains, it is also significantly lighter, using as low as 5.6% of the original network's parameters for similar performance.
2020-05-12
Proceedings of the Conference on Robot Learning (publié)
proceedings.mlr.press
arxiv.org
Differential neural circuitry behind autism subtypes with imbalanced social-communicative and restricted repetitive behavior symptoms
Natasha Bertelsen
Isotta Landi
Richard A.I. Bethlehem
Jakob Seidlitz
Elena Maria Busuoli
Veronica Mandelli
Eleonora Satta
Stavros Trakoshis
Bonnie Auyeung
Prantik Kundu
Eva Loth
Guillaume Dumas
Sarah Baumeister
Christian Beckmann
Sven Bölte
Thomas Bourgeron
Tony Charman
Sarah Durston
Christine Ecker
Rosemary Holt … (voir 15 de plus)
Mark Johnson
Emily J. H. Jones
Luke Mason
Andreas Meyer-Lindenberg
Carolin Moessnang
Marianne Oldehinkel
Antonio Persico
Julian Tillmann
Steven C. R. Williams
Will Spooren
Declan Murphy
Jan K. Buitelaar
Simon Baron-Cohen
Meng-Chuan Lai
Michael V. Lombardo
Social-communication (SC) and restricted repetitive behaviors (RRB) are autism diagnostic symptom domains. SC and RRB severity can markedly … (voir plus)differ within and between individuals and may be underpinned by different neural circuitry and genetic mechanisms. Modeling SC-RRB balance could help identify how neural circuitry and genetic mechanisms map onto such phenotypic heterogeneity. Here we developed a phenotypic stratification model that makes highly accurate (97-99%) out-of-sample SC=RRB, SC>RRB, and RRB>SC subtype predictions. Applying this model to resting state fMRI data from the EU-AIMS LEAP dataset (n=509), we find that while the phenotypic subtypes share many commonalities in terms of intrinsic functional connectivity, they also show subtype-specific qualitative differences compared to a typically-developing group (TD). Specifically, the somatomotor network is hypoconnected with perisylvian circuitry in SC>RRB and visual association circuitry in SC=RRB. The SC=RRB subtype also showed hyperconnectivity between medial motor and anterior salience circuitry. Genes that are highly expressed within these subtype-specific networks show a differential enrichment pattern with known ASD associated genes, indicating that such circuits are affected by differing autism-associated genomic mechanisms. These results suggest that SC-RRB imbalance subtypes share some commonalities but also express subtle differences in functional neural circuitry and the genomic underpinnings behind such circuitry.
2020-05-10
bioRxiv (prépublication)
doi.org