Publications

A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering
Tegan Maharaj
Nicolas Ballas
Anna Rohrbach
Aaron Courville
Christopher Pal
While deep convolutional neural networks frequently approach or exceed human-level performance at benchmark tasks involving static images, e… (voir plus)xtending this success to moving images is not straightforward. Having models which can learn to understand video is of interest for many applications, including content recommendation, prediction, summarization, event/object detection and understanding human visual perception, but many domains lack sufficient data to explore and perfect video models. In order to address the need for a simple, quantitative benchmark for developing and understanding video, we present MovieFIB, a fill-in-the-blank question-answering dataset with over 300,000 examples, based on descriptive video annotations for the visually impaired. In addition to presenting statistics and a description of the dataset, we perform a detailed analysis of 5 different models' predictions, and compare these with human performance. We investigate the relative importance of language, static (2D) visual features, and moving (3D) visual features; the effects of increasing dataset size, the number of frames sampled; and of vocabulary size. We illustrate that: this task is not solvable by a language model alone; our model combining 2D and 3D visual information indeed provides the best result; all models perform significantly worse than human-level. We provide human evaluations for responses given by different models and find that accuracy on the MovieFIB evaluation corresponds well with human judgement. We suggest avenues for improving video models, and hope that the proposed dataset can be useful for measuring and encouraging progress in this very interesting field.
2017-07-20
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (publié)
doi.org
arxiv.org
GuessWhat?! Visual Object Discovery through Multi-modal Dialogue
Harm de Vries
Florian Strub
A. Chandar
Olivier Pietquin
Hugo Larochelle
Aaron Courville
We introduce GuessWhat?!, a two-player guessing game as a testbed for research on the interplay of computer vision and dialogue systems. The… (voir plus) goal of the game is to locate an unknown object in a rich image scene by asking a sequence of questions. Higher-level image understanding, like spatial reasoning and language grounding, is required to solve the proposed task. Our key contribution is the collection of a large-scale dataset consisting of 150K human-played games with a total of 800K visual question-answer pairs on 66K images. We explain our design decisions in collecting the dataset and introduce the oracle and questioner tasks that are associated with the two players of the game. We prototyped deep learning models to establish initial baselines of the introduced tasks.
2017-07-20
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (publié)
doi.org
arxiv.org
Prediction of Extubation readiness in extremely preterm infants by the automated analysis of cardiorespiratory behavior: study protocol
Wissam Shalish
Lara J. Kanbar
Smita Rao
Carlos A. Robles-Rubio
Lajos Kovacs
Sanjay Chawla
Martin Keszler
Doina Precup
Karen Brown
Robert E. Kearney
Guilherme M. Sant’Anna
BackgroundExtremely preterm infants (≤ 28 weeks gestation) commonly require endotracheal intubation and mechanical ventilation (MV) to ma… (voir plus)intain adequate oxygenation and gas exchange. Given that MV is independently associated with important adverse outcomes, efforts should be made to limit its duration. However, current methods for determining extubation readiness are inaccurate and a significant number of infants fail extubation and require reintubation, an intervention that may be associated with increased morbidities. A variety of objective measures have been proposed to better define the optimal time for extubation, but none have proven clinically useful. In a pilot study, investigators from this group have shown promising results from sophisticated, automated analyses of cardiorespiratory signals as a predictor of extubation readiness. The aim of this study is to develop an automated predictor of extubation readiness using a combination of clinical tools along with novel and automated measures of cardiorespiratory behavior, to assist clinicians in determining when extremely preterm infants are ready for extubation.MethodsIn this prospective, multicenter observational study, cardiorespiratory signals will be recorded from 250 eligible extremely preterm infants with birth weights ≤1250 g immediately prior to their first planned extubation. Automated signal analysis algorithms will compute a variety of metrics for each infant, and machine learning methods will then be used to find the optimal combination of these metrics together with clinical variables that provide the best overall prediction of extubation readiness. Using these results, investigators will develop an Automated system for Prediction of EXtubation (APEX) readiness that will integrate the software for data acquisition, signal analysis, and outcome prediction into a single application suitable for use by medical personnel in the neonatal intensive care unit. The performance of APEX will later be prospectively validated in 50 additional infants.DiscussionThe results of this research will provide the quantitative evidence needed to assist clinicians in determining when to extubate a preterm infant with the highest probability of success, and could produce significant improvements in extubation outcomes in this population.Trial registrationClinicaltrials.gov identifier: NCT01909947. Registered on July 17 2013.Trial sponsor: Canadian Institutes of Health Research (CIHR).
2017-07-16
BMC Pediatrics (publié)
doi.org
Unimodal Probability Distributions for Deep Ordinal Classification
Christopher Beckham
Christopher Pal
Probability distributions produced by the cross-entropy loss for ordinal classification problems can possess undesired properties. We propos… (voir plus)e a straightforward technique to constrain discrete ordinal probability distributions to be unimodal via a combination of the Poisson probability mass function and the softmax nonlinearity. We evaluate this approach on two large ordinal image datasets and obtain promising results.
2017-07-16
Proceedings of the 34th International Conference on Machine Learning (publié)
proceedings.mlr.press
proceedings.mlr.press
A Semi-Markov Chain Approach to Modeling Respiratory Patterns Prior to Extubation in Preterm Infants
Charles C. Onu
Lara J. Kanbar
Wissam Shalish
Karen A. Brown
Guilherme M. Sant'Anna
Robert E. Kearney
Doina Precup
After birth, extremely preterm infants often require specialized respiratory management in the form of invasive mechanical ventilation (IMV)… (voir plus). Protracted IMV is associated with detrimental outcomes and morbidities. Premature extubation, on the other hand, would necessitate reintubation which is risky, technically challenging and could further lead to lung injury or disease. We present an approach to modeling respiratory patterns of infants who succeeded extubation and those who required reintubation which relies on Markov models. We compare the use of traditional Markov chains to semi-Markov models which emphasize cross-pattern transitions and timing information, and to multi-chain Markov models which can concisely represent non-stationarity in respiratory behavior over time. The models we developed expose specific, unique similarities as well as vital differences between the two populations.
2017-07-10
2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (publié)
doi.org
arxiv.org
Multiscale sequence modeling with a learned dictionary
Bart Van Merriënboer
Amartya Sanyal
Hugo Larochelle
Yoshua Bengio
We propose a generalization of neural network sequence models. Instead of predicting one symbol at a time, our multi-scale model makes predi… (voir plus)ctions over multiple, potentially overlapping multi-symbol tokens. A variation of the byte-pair encoding (BPE) compression algorithm is used to learn the dictionary of tokens that the model is trained with. When applied to language modelling, our model has the flexibility of character-level models while maintaining many of the performance benefits of word-level models. Our experiments show that this model performs better than a regular LSTM on language modeling tasks, especially for smaller models.
2017-07-02
ArXiv (prépublication)
arxiv.org
arxiv.org
Detecting Large Concept Extensions for Conceptual Analysis
L. Chartrand
Jackie CK Cheung
Mohamed Bouguessa
2017-07-01
Machine Learning and Data Mining in Pattern Recognition (publié)
doi.org
arxiv.org
Variance Regularizing Adversarial Learning
Karan Grewal
R Devon Hjelm
Yoshua Bengio
We study how, in generative adversarial networks, variance in the discriminator's output affects the generator's ability to learn the data d… (voir plus)istribution. In particular, we contrast the results from various well-known techniques for training GANs when the discriminator is near-optimal and updated multiple times per update to the generator. As an alternative, we propose an additional method to train GANs by explicitly modeling the discriminator's output as a bi-modal Gaussian distribution over the real/fake indicator variables. In order to do this, we train the Gaussian classifier to match the target bi-modal distribution implicitly through meta-adversarial training. We observe that our new method, when trained together with a strong discriminator, provides meaningful, non-vanishing gradients.
2017-07-01
ArXiv (prépublication)
openreview.net
openreview.net
APEX_SCOPE: A graphical user interface for visualization of multi-modal data in inter-disciplinary studies.
Lara J. Kanbar
Wissam Shalish
Doina Precup
Karen A. Brown
Guilherme M. Sant'Anna
Robert E. Kearney
2017-06-30
Annual International Conference of the IEEE Engineering in Medicine and Biology Society (publié)
doi.org
Towards more hardware-friendly deep learning
Yoshua Bengio
2017-06-23
TiML '17 (publié)
doi.org
Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control
Riashat Islam
Peter Henderson
Maziar Gomrokchi
Doina Precup
Policy gradient methods in reinforcement learning have become increasingly prevalent for state-of-the-art performance in continuous control … (voir plus)tasks. Novel methods typically benchmark against a few key algorithms such as deep deterministic policy gradients and trust region policy optimization. As such, it is important to present and use consistent baselines experiments. However, this can be difficult due to general variance in the algorithms, hyper-parameter tuning, and environment stochasticity. We investigate and discuss: the significance of hyper-parameters in policy gradients for continuous control, general variance in the algorithms, and reproducibility of reported results. We provide guidelines on reporting novel results as comparisons against baseline methods such that future researchers can make informed decisions when investigating novel methods.
2017-06-16
ICML.cc/2017/RML (poster)
openreview.net
openreview.net
Time-Varying Mixtures of Markov Chains: An Application to Road Traffic Modeling
Sean Lawlor
Michael G. Rabbat
Time-varying mixture models are useful for representing complex, dynamic distributions. Components in the mixture model can appear and disap… (voir plus)pear, and persisting components can evolve. This allows great flexibility in streaming data applications where the model can be adjusted as new data arrives. Fitting a mixture model with computational guarantees which can meet real-time requirements is challenging with existing algorithms, especially when the model order can vary with time. Existing approximate inference methods may require multiple restarts to search for a good local solution. Monte-Carlo methods can be used to jointly estimate the model order and model parameters, but when the distribution of each mixand has a high-dimensional parameter space, they suffer from the curse of dimensionality and and from slow convergence. This paper proposes a generative model for time-varying mixture models, tailored for mixtures of discrete-time Markov chains. A novel, deterministic inference procedure is introduced and is shown to be suitable for applications requiring real-time estimation, and the method is guaranteed to converge at each time step. As a motivating application, we model and predict traffic patterns in a transportation network. Experiments illustrate the performance of the scheme and offer insights regarding tuning of the algorithm parameters. The experiments also investigate the predictive power of the proposed model compared to less complex models and demonstrate the superiority of the mixture model approach for prediction of traffic routes in real data.
2017-06-14
IEEE Transactions on Signal Processing (publié)
doi.org