Portrait of Chinnadhurai Sankar is unavailable

Chinnadhurai Sankar

Alumni

Publications

The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach
Iulian Vlad Serban
Chinnadhurai Sankar
Michael Pieper
Joelle Pineau
Yoshua Bengio
Deep reinforcement learning has recently shown many impressive successes. However, one major obstacle towards applying such methods to real-… (see more)world problems is their lack of data-efficiency. To this end, we propose the Bottleneck Simulator: a model-based reinforcement learning method which combines a learned, factorized transition model of the environment with rollout simulations to learn an effective policy from few examples. The learned transition model employs an abstract, discrete (bottleneck) state, which increases sample efficiency by reducing the number of model parameters and by exploiting structural properties of the environment. We provide a mathematical analysis of the Bottleneck Simulator in terms of fixed points of the learned policy, which reveals how performance is affected by four distinct sources of error: an error related to the abstract space structure, an error related to the transition model estimation variance, an error related to the transition model estimation bias, and an error related to the transition model class bias. Finally, we evaluate the Bottleneck Simulator on two natural language processing tasks: a text adventure game and a real-world, complex dialogue response selection task. On both tasks, the Bottleneck Simulator yields excellent performance beating competing approaches.
2020-10-26
Journal of Artificial Intelligence Research (published)
doi.org
arxiv.org
Towards Non-Saturating Recurrent Units for Modelling Long-Term Dependencies
Sarath Chandar
Chinnadhurai Sankar
Eugene Vorontsov
Samira Ebrahimi Kahou
Yoshua Bengio
Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish durin… (see more)g training, as the sequence length increases. Gradients can be attenuated by transition operators and are attenuated or dropped by activation functions. Canonical architectures like LSTM alleviate this issue by skipping information through a memory mechanism. We propose a new recurrent architecture (Non-saturating Recurrent Unit; NRU) that relies on a memory mechanism but forgoes both saturating activation functions and saturating gates, in order to further alleviate vanishing gradients. In a series of synthetic and real world tasks, we demonstrate that the proposed model is the only model that performs among the top 2 models across all tasks with and without long-term dependencies, when compared against a range of other architectures.
2019-07-16
Proceedings of the AAAI Conference on Artificial Intelligence (published)
doi.org
arxiv.org
Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study
Chinnadhurai Sankar
Sandeep Subramanian
Christopher Pal
Sarath Chandar
Yoshua Bengio
Neural generative models have been become increasingly popular when building conversational agents. They offer flexibility, can be easily ad… (see more)apted to new domains, and require minimal domain engineering. A common criticism of these systems is that they seldom understand or use the available dialog history effectively. In this paper, we take an empirical approach to understanding how these models use the available dialog history by studying the sensitivity of the models to artificially introduced unnatural changes or perturbations to their context at test time. We experiment with 10 different types of perturbations on 4 multi-turn dialog datasets and find that commonly used neural dialog architectures like recurrent and transformer-based seq2seq models are rarely sensitive to most perturbations such as missing or reordering utterances, shuffling words, etc. Also, by open-sourcing our code, we believe that it will serve as a useful diagnostic tool for evaluating dialog systems in the future.
2019-06-30
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (published)
doi.org
arxiv.org
A Deep Reinforcement Learning Chatbot (Short Version)
Iulian V. Serban
Chinnadhurai Sankar
Mathieu Germain
Saizheng Zhang
Zhouhan Lin
Sandeep Subramanian
Taesup Kim
Michael Pieper
Sarath Chandar
Nan Rosemary Ke
Sai Rajeswar
Alexandre De Brébisson
Jose M. R. Sotelo
Dendi Suhubdy
Vincent Michalski
Alexandre Nguyen
Joelle Pineau
Yoshua Bengio
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon … (see more)Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including neural network and template-based models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than other systems. The results highlight the potential of coupling ensemble systems with deep reinforcement learning as a fruitful path for developing real-world, open-domain conversational agents.
2017-12-31
arXiv (preprint)
doi.org
arxiv.org
A Deep Reinforcement Learning Chatbot
Iulian V. Serban
Chinnadhurai Sankar
Mathieu Germain
Saizheng Zhang
Zhouhan Lin
Sandeep Subramanian
Taesup Kim
Michael Pieper
Sarath Chandar
Nan Rosemary Ke
Sai Mudumba
Alexandre De Brébisson
Jose M. R. Sotelo
Dendi Suhubdy
Vincent Michalski
Alexandre Nguyen
Joelle Pineau
Yoshua Bengio
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon … (see more)Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including template-based models, bag-of-words models, sequence-to-sequence neural network and latent variable neural network models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than many competing systems. Due to its machine learning architecture, the system is likely to improve with additional data.
2017-09-06
ArXiv (preprint)
doi.org
arxiv.org
Variance Reduction in SGD by Distributed Importance Sampling
Guillaume Alain
Alex Lamb
Chinnadhurai Sankar
Aaron Courville
Yoshua Bengio
Humans are able to accelerate their learning by selecting training materials that are the most informative and at the appropriate level of d… (see more)ifficulty. We propose a framework for distributing deep learning in which one set of workers search for the most informative examples in parallel while a single worker updates the model on examples selected by importance sampling. This leads the model to update using an unbiased estimate of the gradient which also has minimum variance when the sampling proposal is proportional to the L2-norm of the gradient. We show experimentally that this method reduces gradient variance even in a context where the cost of synchronization across machines cannot be ignored, and where the factors for importance sampling are not updated instantly across the training set.
2016-02-16
ICLR.cc/2016/workshop (unknown)
openreview.net
openreview.net