Publications

Active 3D Shape Reconstruction from Vision and Touch
Edward J. Smith
David Meger
Luis Pineda
Roberto Calandra
Jitendra Malik
Adriana Romero
Michal Drozdzal
Humans build 3D understandings of the world through active object exploration, using jointly their senses of vision and touch. However, in 3… (see more)D shape reconstruction, most recent progress has relied on static datasets of limited sensory data such as RGB images, depth maps or haptic readings, leaving the active exploration of the shape largely unexplored. In active touch sensing for 3D reconstruction, the goal is to actively select the tactile readings that maximize the improvement in shape reconstruction accuracy. However, the development of deep learning-based active touch models is largely limited by the lack of frameworks for shape exploration. In this paper, we focus on this problem and introduce a system composed of: 1) a haptic simulator leveraging high spatial resolution vision-based tactile sensors for active touching of 3D objects; 2) a mesh-based 3D shape reconstruction model that relies on tactile or visuotactile signals; and 3) a set of data-driven solutions with either tactile or visuotactile priors to guide the shape exploration. Our framework enables the development of the first fully data-driven solutions to active touch on top of learned models for object understanding. Our experiments show the benefits of such solutions in the task of 3D shape understanding where our models consistently outperform natural baselines. We provide our framework as a tool to foster future research in this direction.
2021-11-08
NeurIPS.cc/2021/Conference (poster)
openreview.net
openreview.net
MICo: Improved representations via sampling-based state similarity for Markov decision processes
Pablo Samuel Castro
Tyler Kastner
Prakash Panangaden
Mark Rowland
We present a new behavioural distance over the state space of a Markov decision process, and demonstrate the use of this distance as an effe… (see more)ctive means of shaping the learnt representations of deep reinforcement learning agents. While existing notions of state similarity are typically difficult to learn at scale due to high computational cost and lack of sample-based algorithms, our newly-proposed distance addresses both of these issues. In addition to providing detailed theoretical analyses, we provide empirical evidence that learning this distance alongside the value function yields structured and informative representations, including strong results on the Arcade Learning Environment benchmark.
2021-11-08
NeurIPS.cc/2021/Conference (poster)
openreview.net
openreview.net
Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPs
Harsh Satija
Philip S. Thomas
Joelle Pineau
Romain Laroche
We study the problem of Safe Policy Improvement (SPI) under constraints in the offline Reinforcement Learning (RL) setting. We consider the … (see more)scenario where: (i) we have a dataset collected under a known baseline policy, (ii) multiple reward signals are received from the environment inducing as many objectives to optimize. We present an SPI formulation for this RL setting that takes into account the preferences of the algorithm’s user for handling the trade-offs for different reward signals while ensuring that the new policy performs at least as well as the baseline policy along each individual objective. We build on traditional SPI algorithms and propose a novel method based on Safe Policy Iteration with Baseline Bootstrapping (SPIBB, Laroche et al., 2019) that provides high probability guarantees on the performance of the agent in the true environment. We show the effectiveness of our method on a synthetic grid-world safety task as well as in a real-world critical care context to learn a policy for the administration of IV fluids and vasopressors to treat sepsis.
2021-11-08
NeurIPS.cc/2021/Conference (poster)
openreview.net
openreview.net
Temporally Abstract Partial Models
Khimya Khetarpal
Zafarali Ahmed
Gheorghe Comanici
Doina Precup
Humans and animals have the ability to reason and make predictions about different courses of action at many time scales. In reinforcement l… (see more)earning, option models (Sutton, Precup \& Singh, 1999; Precup, 2000) provide the framework for this kind of temporally abstract prediction and reasoning. Natural intelligent agents are also able to focus their attention on courses of action that are relevant or feasible in a given situation, sometimes termed affordable actions. In this paper, we define a notion of affordances for options, and develop temporally abstract partial option models, that take into account the fact that an option might be affordable only in certain situations. We analyze the trade-offs between estimation and approximation error in planning and learning when using such models, and identify some interesting special cases. Additionally, we demonstrate empirically the potential impact of partial option models on the efficiency of planning.
2021-11-08
NeurIPS.cc/2021/Conference (poster)
doi.org
openreview.net
On the Expressivity of Markov Reward
David Abel
Will Dabney
Anna Harutyunyan
Mark K. Ho
Michael L. Littman
Doina Precup
Satinder Singh
Reward is the driving force for reinforcement-learning agents. This paper is dedicated to understanding the expressivity of reward as a way … (see more)to capture tasks that we would want an agent to perform. We frame this study around three new abstract notions of"task"that might be desirable: (1) a set of acceptable behaviors, (2) a partial ordering over behaviors, or (3) a partial ordering over trajectories. Our main results prove that while reward can express many of these tasks, there exist instances of each task type that no Markov reward function can capture. We then provide a set of polynomial-time algorithms that construct a Markov reward function that allows an agent to optimize tasks of each of these three types, and correctly determine when no such reward function exists. We conclude with an empirical study that corroborates and illustrates our theoretical findings.
2021-11-08
NeurIPS.cc/2021/Conference (oral)
openreview.net
openreview.net
A Variational Perspective on Diffusion-Based Generative Models and Score Matching
Chin-wei Huang
Jae Hyun Lim
Aaron Courville
Discrete-time diffusion-based generative models and score matching methods have shown promising results in modeling high-dimensional image d… (see more)ata. Recently, Song et al. (2021) show that diffusion processes that transform data into noise can be reversed via learning the score function, i.e. the gradient of the log-density of the perturbed data. They propose to plug the learned score function into an inverse formula to define a generative diffusion process. Despite the empirical success, a theoretical underpinning of this procedure is still lacking. In this work, we approach the (continuous-time) generative diffusion directly and derive a variational framework for likelihood estimation, which includes continuous-time normalizing flows as a special case, and can be seen as an infinitely deep variational autoencoder. Under this framework, we show that minimizing the score-matching loss is equivalent to maximizing a lower bound of the likelihood of the plug-in reverse SDE proposed by Song et al. (2021), bridging the theoretical gap.
2021-11-08
NeurIPS.cc/2021/Conference (spotlight)
openreview.net
openreview.net
Problèmes associés au déploiement des modèles fondés sur l’apprentissage machine en santé
Joseph Paul Cohen
Tianshi Cao
Joseph D Viviano
Chin-wei Huang
Michael Fralick
Marzyeh Ghassemi
Muhammad Mamdani
Russell Greiner
Yoshua Bengio
2021-11-06
Canadian Medical Association Journal (published)
doi.org
Learned Image Compression for Machine Perception
Felipe Codevilla
Jean Gabriel Simard
Ross Goroshin
Chris Pal
Recent work has shown that learned image compression strategies can outperform standard hand-crafted compression algorithms that have been d… (see more)eveloped over decades of intensive research on the rate-distortion trade-off. With growing applications of computer vision, high quality image reconstruction from a compressible representation is often a secondary objective. Compression that ensures high accuracy on computer vision tasks such as image segmentation, classification, and detection therefore has the potential for significant impact across a wide variety of settings. In this work, we develop a framework that produces a compression format suitable for both human perception and machine perception. We show that representations can be learned that simultaneously optimize for compression and performance on core vision tasks. Our approach allows models to be trained directly from compressed representations, and this approach yields increased performance on new tasks and in low-shot learning settings. We present results that improve upon segmentation and detection performance compared to standard high quality JPGs, but with representations that are four to ten times smaller in terms of bits per pixel. Further, unlike naive compression methods, at a level ten times smaller than standard JEPGs, segmentation and detection models trained from our format suffer only minor degradation in performance.
2021-11-02
ArXiv (preprint)
doi.org
arxiv.org
Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval
Devang Kulshreshtha
Robert Belfer
Iulian V. Serban
Siva Reddy
In this work, we introduce back-training, an alternative to self-training for unsupervised domain adaptation (UDA). While self-training gene… (see more)rates synthetic training data where natural inputs are aligned with noisy outputs, back-training results in natural outputs aligned with noisy inputs. This significantly reduces the gap between target domain and synthetic data distribution, and reduces model overfitting to source domain. We run UDA experiments on question generation and passage retrieval from the Natural Questions domain to machine learning and biomedical domains. We find that back-training vastly outperforms self-training by a mean improvement of 7.8 BLEU-4 points on generation, and 17.6% top-20 retrieval accuracy across both domains. We further propose consistency filters to remove low-quality synthetic data before training. We also release a new domain-adaptation dataset - MLQuestions containing 35K unaligned questions, 50K unaligned passages, and 3K aligned question-passage pairs.
2021-10-31
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (published)
doi.org
arxiv.org
#EEGManyLabs: Investigating the replicability of influential EEG experiments
Yuri G Pavlov
Nika Adamian
Stefan Appelhoff
Mahnaz Arvaneh
Christopher Benwell
Christian Beste
Amy Bland
Daniel E. Bradford
Florian Bublatzky
Niko A. Busch
Peter E. Clayson
Damian Cruse
Artur Czeszumski
Anna Dreber
Guillaume Dumas
Benedikt Valerian Ehinger
Giorgio Ganis
Xun He
José Antonio Hinojosa
Christoph Huber-Huber … (see 39 more)
Michael Inzlicht
Bradley N Jack
Magnus Johannesson
Rhiannon Jones
Evgenii Kalenkovich
Laura Kaltwasser
Hamid Karimi-Rouzbahani
Andreas Keil
Peter König
Layla Kouara
Louisa Kulke
Cecile D. Ladouceur
Nicolas Langer
Heinrich R. Liesefeld
David Luque
Annmarie MacNamara
Liad Mudrik
Muthuraman Muthuraman
Lauren Browning Neal
Gustav Nilsonne
Guiomar Niso
Sebastian Ocklenburg
Robert Oostenveld
Cyril R. Pernet
Gilles Pourtois
Manuela Ruzzoli
Sarah Sass
Alexandre Schaefer
Magdalena Senderecka
Joel S. Snyder
Christian K. Tamnes
E Tognoli
Marieke K. van Vugt
Edelyn Verona
Robin Vloeberghs
Dominik Welke
Jan Wessel
Ilya Zakharov
Faisal Mushtaq
2021-10-31
Cortex (published)
doi.org
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little
Koustuv Sinha
Robin Jia
Dieuwke Hupkes
Joelle Pineau
Adina Williams
Douwe Kiela
A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to repres… (see more)ent the syntactic structures prevalent in classical NLP pipelines. In this paper, we propose a different explanation: MLMs succeed on downstream tasks almost entirely due to their ability to model higher-order word co-occurrence statistics. To demonstrate this, we pre-train MLMs on sentences with randomly shuffled word order, and show that these models still achieve high accuracy after fine-tuning on many downstream tasks—including tasks specifically designed to be challenging for models that ignore word order. Our models perform surprisingly well according to some parametric syntactic probes, indicating possible deficiencies in how we test representations for syntactic information. Overall, our results show that purely distributional information largely explains the success of pre-training, and underscore the importance of curating challenging evaluation datasets that require deeper linguistic knowledge.
2021-10-31
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (published)
doi.org
arxiv.org
Opioid prescribing among new users for non-cancer pain in the USA, Canada, UK, and Taiwan: A population-based cohort study
Meghna Jani
Nadyne Girard
David W. Bates
David L Buckeridge
Therese Sheppard
Jack Li
Usman Iqbal
Shelly Vik
Colin Weaver
Judy Seidel
William G. Dixon
Robyn Tamblyn
Background The opioid epidemic in North America has been driven by an increase in the use and potency of prescription opioids, with ensuing … (see more)excessive opioid-related deaths. Internationally, there are lower rates of opioid-related mortality, possibly because of differences in prescribing and health system policies. Our aim was to compare opioid prescribing rates in patients without cancer, across 5 centers in 4 countries. In addition, we evaluated differences in the type, strength, and starting dose of medication and whether these characteristics changed over time. Methods and findings We conducted a retrospective multicenter cohort study of adults who are new users of opioids without prior cancer. Electronic health records and administrative health records from Boston (United States), Quebec and Alberta (Canada), United Kingdom, and Taiwan were used to identify patients between 2006 and 2015. Standard dosages in morphine milligram equivalents (MMEs) were calculated according to The Centers for Disease Control and Prevention. Age- and sex-standardized opioid prescribing rates were calculated for each jurisdiction. Of the 2,542,890 patients included, 44,690 were from Boston (US), 1,420,136 Alberta, 26,871 Quebec (Canada), 1,012,939 UK, and 38,254 Taiwan. The highest standardized opioid prescribing rates in 2014 were observed in Alberta at 66/1,000 persons compared to 52, 51, and 18/1,000 in the UK, US, and Quebec, respectively. The median MME/day (IQR) at initiation was highest in Boston at 38 (20 to 45); followed by Quebec, 27 (18 to 43); Alberta, 23 (9 to 38); UK, 12 (7 to 20); and Taiwan, 8 (4 to 11). Oxycodone was the first prescribed opioid in 65% of patients in the US cohort compared to 14% in Quebec, 4% in Alberta, 0.1% in the UK, and none in Taiwan. One of the limitations was that data were not available from all centers for the entirety of the 10-year period. Conclusions In this study, we observed substantial differences in opioid prescribing practices for non-cancer pain between jurisdictions. The preference to start patients on higher MME/day and more potent opioids in North America may be a contributing cause to the opioid epidemic.
2021-10-31
PLoS Medicine (published)
doi.org