Automated curriculum generation for Policy Gradients from Demonstrations
Anirudh Srinivasan
Dzmitry Bahdanau
Maxime Chevalier-Boisvert
Yoshua Bengio
2019-12-01
ArXiv (preprint)
arxiv.org
arxiv.org
Expressiveness and Learning of Hidden Quantum Markov Models
Sandesh M. Adhikary
Siddarth Srinivasan
Geoff Gordon
Byron Boots
Extending classical probabilistic reasoning using the quantum mechanical view of probability has been of recent interest, particularly in th… (see more)e development of hidden quantum Markov models (HQMMs) to model stochastic processes. However, there has been little progress in characterizing the expressiveness of such models and learning them from data. We tackle these problems by showing that HQMMs are a special subclass of the general class of observable operator models (OOMs) that do not suffer from the \emph{negative probability problem} by design. We also provide a feasible retraction-based learning algorithm for HQMMs using constrained gradient descent on the Stiefel manifold of model parameters. We demonstrate that this approach is faster and scales to larger models than previous learning algorithms.
2019-12-01
ArXiv (preprint)
arxiv.org
arxiv.org
Forgetting at biologically realistic levels of neurogenesis in a large-scale hippocampal model
Lina M. Tran
Sheena A. Josselyn
Blake Richards
Paul W. Frankland
2019-12-01
Behavioural Brain Research (published)
doi.org
On generalized surrogate duality in mixed-integer nonlinear programming
Benjamin Müller
Gonzalo Muñoz
Maxime Gasse
Ambros Gleixner
Andrea Lodi
Felipe Serrano
2019-12-01
ArXiv (preprint)
doi.org
arxiv.org
Networked control of coupled subsystems: Spectral decomposition and low-dimensional solutions
Shuang Gao
Aditya Mahajan
In this paper, we investigate optimal networked control of coupled subsystems where the dynamics and the cost couplings depend on an underly… (see more)ing weighted graph. We use the spectral decomposition of the graph adjacency matrix to decompose the overall system into (L+1) systems with decoupled dynamics and cost, where L is the rank of the adjacency matrix. Consequently, the optimal control input at each subsystem can be computed by solving (L+1) decoupled Riccati equations. A salient feature of the result is that the solution complexity depends on the rank of the adjacency matrix rather than the size of the network (i.e., the number of nodes). Therefore, the proposed solution framework provides a scalable method for synthesizing and implementing optimal control laws for large-scale systems.
2019-12-01
IEEE Conference on Decision and Control (published)
doi.org
Restless bandits with controlled restarts: Indexability and computation of Whittle index
Nima Akbarzadeh
Aditya Mahajan
Motivated by applications in machine repair, queueing, surveillance, and clinic care, we consider a scheduling problem where a decision make… (see more)r can reset m out of n Markov processes at each time. Processes that are reset, restart according to a known probability distribution and processes that are not reset, evolve in a Markovian manner. Due to the high complexity of finding an optimal policy, such scheduling problems are often modeled as restless bandits. We show that the model satisfies a technical condition known as indexability. For indexable restless bandits, the Whittle index policy, which computes a function known as Whittle index for each process and resets the m processes with the lowest index, is known to be a good heuristic. The Whittle index is computed by solving an auxiliary Markov decision problem for each arm. When the optimal policy for this auxiliary problem is threshold based, we use ideas from renewal theory to derive closed form expression for the Whittle index. We present detailed numerical experiments which suggest that Whittle index policy performs close to the optimal policy and performs significantly better than myopic policy, which is a commonly used heuristic.
2019-12-01
IEEE Conference on Decision and Control (published)
doi.org
Deconstructing and reconstructing word embedding algorithms
Edward Daniel Newell
Kian Kenyon-Dean
Jackie Cheung
Uncontextualized word embeddings are reliable feature representations of words used to obtain high quality results for various NLP applicati… (see more)ons. Given the historical success of word embeddings in NLP, we propose a retrospective on some of the most well-known word embedding algorithms. In this work, we deconstruct Word2vec, GloVe, and others, into a common form, unveiling some of the necessary and sufficient conditions required for making performant word embeddings. We find that each algorithm: (1) fits vector-covector dot products to approximate pointwise mutual information (PMI); and, (2) modulates the loss gradient to balance weak and strong signals. We demonstrate that these two algorithmic features are sufficient conditions to construct a novel word embedding algorithm, Hilbert-MLE. We find that its embeddings obtain equivalent or better performance against other algorithms across 17 intrinsic and extrinsic datasets.
2019-11-29
ArXiv (preprint)
arxiv.org
arxiv.org
Driver perceptions of advanced driver assistance systems and safety
Sophie Le Page
Jason Millar
Kelly Selina Bronson
Shalaleh Rismani
AJung Moon
Advanced driver assistance systems (ADAS) are often used in the automotive industry to highlight innovative improvements in vehicle safety. … (see more)However, today it is unclear whether certain automation (e.g., adaptive cruise control, lane keeping, parking assist) increases safety of our roads. In this paper, we investigate driver awareness, use, perceived safety, knowledge, training, and attitudes toward ADAS with different automation systems/features. Results of our online survey (n=1018) reveal that there is a significant difference in frequency of use and perceived safety for different ADAS features. Furthermore, we find that at least 70% of drivers activate an ADAS feature"most or all of the time"when driving, yet we find that at least 40% of drivers report feeling that ADAS often compromises their safety when activated. We also find that most respondents learn how to use ADAS in their vehicles by trying it out on the road by themselves, rather than through any formal driver education and training. These results may mirror how certain ADAS features are often activated by default resulting in high usage rates. These results also suggest a lack of driver training and education for safely interacting with, and operating, ADAS, such as turning off systems/features. These findings contribute to a critical discussion about the overall safety implications of current ADAS, especially as they enable higher-level automation features to creep into personal vehicles without a lockstep response in training, regulation, and policy.
2019-11-25
ArXiv (preprint)
arxiv.org
arxiv.org
Drivers' Awareness, Knowledge, and Use of Autonomous Driving Assistance Systems (ADAS) and Vehicle Automation
Kelly Selina Bronson
Sophie Le Page
Katherine M. Robinson
AJung Moon
Shalaleh Rismani
Jason Millar
Advanced driver assistance systems (ADAS) technologies in vehicles (e.g. park assist, lane change assist, emergency braking, etc.), which ta… (see more)ke over parts of the driving task of human drivers, are advancing at a disruptive pace and hold the potential to deliver many benefits to society. However, public understanding of ADAS systems, and driver training and licensing for using them, are lagging behind the fast-paced technological development, which could raise safety issues or slow the deployment of ADAS, thus offsetting their potential benefits. There is, therefore, a need to investigate issues related to public perception of ADAS in order to develop appropriate policies and governance structures which support innovation, and result in the smooth deployment and acceptance of appropriate ADAS for society. In this work we perform a quantitative public survey to better understand how the public's awareness and knowledge of ADAS technologies in their vehicles correlate to their use or engagement of those technologies. We find that up to 67% of participants never or rarely use optional ADAS in their vehicles (e.g. adaptive cruise control), where women were less likely than men to use ADAS even though women reported more awareness of ADAS in their vehicles, better training, and more willingness to pay for ADAS. By performing this analysis we hope to raise awareness around the public perception of current state-of-the-art in ADAS technologies. We also hope to flag concerns that answers to these questions might raise for the regulatory agencies, and manufacturers, responsible for bringing these technologies to market.
2019-11-25
arXiv.org (preprint)
dblp.uni-trier.de
JANOS: An Integrated Predictive and Prescriptive Modeling Framework
David Bergman
Teng Huang
Philip A. Brooks
Andrea Lodi
A. Raghunathan
Business research practice is witnessing a surge in the integration of predictive modeling and prescriptive analysis. We describe a modeling… (see more) framework JANOS that seamlessly integrates the two streams of analytics, allowing researchers and practitioners to embed machine learning models in an end-to-end optimization framework. JANOS allows for specifying a prescriptive model using standard optimization modeling elements such as constraints and variables. The key novelty lies in providing modeling constructs that enable the specification of commonly used predictive models within an optimization model, have the features of the predictive model as variables in the optimization model, and incorporate the output of the predictive models as part of the objective. The framework considers two sets of decision variables: regular and predicted. The relationship between the regular and the predicted variables is specified by the user as pretrained predictive models. JANOS currently supports linear regression, logistic regression, and neural network with rectified linear activation functions. In this paper, we demonstrate the flexibility of the framework through an example on scholarship allocation in a student enrollment problem and provide a numeric performance evaluation. Summary of Contribution. This paper describes a new software tool, JANOS, that integrates predictive modeling and discrete optimization to assist decision making. Specifically, the proposed solver takes as input user-specified pretrained predictive models and formulates optimization models directly over those predictive models by embedding them within an optimization model through linear transformations.
2019-11-21
ArXiv (preprint)
doi.org
arxiv.org
Deep Active Learning: Unified and Principled Method for Query and Training
Changjian Shui
Fan Zhou
Christian Gagné
Boyu Wang
In this paper, we are proposing a unified and principled method for both the querying and training processes in deep batch active learning. … (see more)We are providing theoretical insights from the intuition of modeling the interactive procedure in active learning as distribution matching, by adopting the Wasserstein distance. As a consequence, we derived a new training loss from the theoretical analysis, which is decomposed into optimizing deep neural network parameters and batch query selection through alternative optimization. In addition, the loss for training a deep neural network is naturally formulated as a min-max optimization problem through leveraging the unlabeled data information. Moreover, the proposed principles also indicate an explicit uncertainty-diversity trade-off in the query batch selection. Finally, we evaluate our proposed method on different benchmarks, consistently showing better empirical performances and a better time-efficient query strategy compared to the baselines.
2019-11-20
ArXiv (preprint)
arxiv.org
arxiv.org
Nonlinear chance-constrained problems with applications to hydro scheduling
Andrea Lodi
Enrico Malaguti
Giacomo Nannicini
Dimitri Thomopulos
2019-11-14
Mathematical Programming (published)
doi.org