Publications

Tensor Regression Networks with various Low-Rank Tensor Approximations
Xingwei Cao
Guillaume Rabusseau
Joelle Pineau
Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposi… (voir plus)ng low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their compressive power, however, the regularization effect of enforcing low-rank tensor structure has not been investigated enough. We study tensor regression networks using various low-rank tensor approximations, aiming to compare the compressive and regularization power of different low-rank constraints. We evaluate the compressive and regularization performances of the proposed model with both deep and shallow convolutional neural networks. The outcome of our experiment suggests the superiority of Global Average Pooling Layer over Tensor Regression Layer when applied to deep convolutional neural network with CIFAR-10 dataset. On the contrary, shallow convolutional neural networks with tensor regression layer and dropout achieved lower test error than both Global Average Pooling and fully-connected layer with dropout function when trained with a small number of samples.
2017-12-27
ArXiv (prépublication)
arxiv.org
arxiv.org
Deep Learning @15 Petaflops/second: Semi-supervised pattern detection for 15 Terabytes of climate data
W. Collins
M. Wehner
M. Prabhat
Thorsten Kurth
Nadathur Satish
Ioannis Mitliagkas
Jian Zhang
Evan Racah
Md. Mostofa Ali Patwary
Narayanan Sundaram
Pradeep Dubey
2017-12-01
(publié)
www.semanticscholar.org
Fast and Flexible Successive-Cancellation List Decoders for Polar Codes
Seyyed Ali Hashemi
Carlo Condo
Warren Gross
Polar codes have gained significant amount of attention during the past few years and have been selected as a coding scheme for the next gen… (voir plus)eration of mobile broadband standard. Among decoding schemes, successive-cancellation list (SCL) decoding provides a reasonable tradeoff between the error-correction performance and hardware implementation complexity when used to decode polar codes, at the cost of limited throughput. The simplified SCL (SSCL) and its extension SSCL-SPC increase the speed of decoding by removing redundant calculations when encountering particular information and frozen bit patterns (rate one and single parity check codes), while keeping the error-correction performance unaltered. In this paper, we improve SSCL and SSCL-SPC by proving that the list size imposes a specific number of path splitting required to decode rate one and single parity check codes. Thus, the number of splitting can be limited while guaranteeing exactly the same error-correction performance as if the paths were forked at each bit estimation. We call the new decoding algorithms Fast-SSCL and Fast-SSCL-SPC. Moreover, we show that the number of path forks in a practical application can be tuned to achieve desirable speed, while keeping the error-correction performance almost unchanged. Hardware architectures implementing both algorithms are then described and implemented: It is shown that our design can achieve
2017-11-01
IEEE Transactions on Signal Processing (publié)
doi.org
arxiv.org
Automatic Differentiation in Myia
Olivier Breuleux
Bart van Merriënboer
Automatic differentiation is an essential feature of machine learning frameworks. However, its implementation in existing frameworks often h… (voir plus)as limitations. In dataflow programming frameworks such as Theano or TensorFlow the representation used makes supporting higher-order gradients difficult. On the other hand, operator overloading frameworks such as PyTorch are flexible, but do not lend themselves well to optimization. With Myia, we attempt to have the best of both worlds: Building on the work by Pearlmutter and Siskind we implement a first-order gradient operator for a subset of the Python programming language.
2017-10-28
(publié)
www.semanticscholar.org
A Multisensor Multi-Bernoulli Filter
Augustin-Alexandru Saucan
Mark Coates
Michael Rabbat
In this paper, we derive a multisensor multi-Bernoulli (MS-MeMBer) filter for multitarget tracking. Measurements from multiple sensors are e… (voir plus)mployed by the proposed filter to update a set of tracks modeled as a multi-Bernoulli random finite set. An exact implementation of the MS-MeMBer update procedure is computationally intractable. We propose an efficient approximate implementation by using a greedy measurement partitioning mechanism. The proposed filter allows for Gaussian mixture or particle filter implementations. Numerical simulations conducted for both linear-Gaussian and nonlinear models highlight the improved accuracy of the MS-MeMBer filter and its reduced computational load with respect to the multisensor cardinalized probability hypothesis density filter and the iterated-corrector cardinality-balanced multi-Bernoulli filter especially for low probabilities of detection.
2017-10-15
IEEE Transactions on Signal Processing (publié)
doi.org
arxiv.org
Learnable Explicit Density for Continuous Latent Space and Variational Inference
Chin-Wei Huang
Ahmed Touati
Laurent Dinh
Michal Drozdzal
Mohammad Havaei
Laurent Charlin
Aaron Courville
In this paper, we study two aspects of the variational autoencoder (VAE): the prior distribution over the latent variables and its correspon… (voir plus)ding posterior. First, we decompose the learning of VAEs into layerwise density estimation, and argue that having a flexible prior is beneficial to both sample generation and inference. Second, we analyze the family of inverse autoregressive flows (inverse AF) and show that with further improvement, inverse AF could be used as universal approximation to any complicated posterior. Our analysis results in a unified approach to parameterizing a VAE, without the need to restrict ourselves to use factorial Gaussians in the latent real space.
2017-10-06
ArXiv (prépublication)
arxiv.org
arxiv.org
Neural Network Based Nonlinear Weighted Finite Automata
Tianyu Li
Guillaume Rabusseau
Doina Precup
Weighted finite automata (WFA) can expressively model functions defined over strings but are inherently linear models. Given the recent succ… (voir plus)esses of nonlinear models in machine learning, it is natural to wonder whether ex-tending WFA to the nonlinear setting would be beneficial. In this paper, we propose a novel model of neural network based nonlinearWFA model (NL-WFA) along with a learning algorithm. Our learning algorithm is inspired by the spectral learning algorithm for WFAand relies on a nonlinear decomposition of the so-called Hankel matrix, by means of an auto-encoder network. The expressive power of NL-WFA and the proposed learning algorithm are assessed on both synthetic and real-world data, showing that NL-WFA can lead to smaller model sizes and infer complex grammatical structures from data.
2017-09-13
ArXiv (prépublication)
arxiv.org
arxiv.org
Predicting Future Disease Activity and Treatment Responders for Multiple Sclerosis Patients Using a Bag-of-Lesions Brain Representation
Andrew Doyle
Doina Precup
Douglas Arnold
Tal Arbel
2017-09-04
Medical Image Computing and Computer Assisted Intervention − MICCAI 2017 (publié)
doi.org
Graphs in Biomedical Image Analysis, Computational Anatomy and Imaging Genetics
M. Cardoso
Tal Arbel
Enzo Ferrante
Xavier Pennec
Adrian Dalca
Sarah Parisot
S. Joshi
Nematollah Batmanghelich
Aristeidis Sotiras
Mads Lenstrup Nielsen
M. Sabuncu
Tom Fletcher
Li Shen
Stanley Durrleman
Stefan H. Sommer
2017-09-01
Lecture Notes in Computer Science (publié)
doi.org
World Knowledge for Reading Comprehension: Rare Entity Prediction with Hierarchical LSTMs Using External Descriptions
Teng Long
Emmanuel Bengio
Ryan Lowe
Jackie Cheung
Doina Precup
Humans interpret texts with respect to some background information, or world knowledge, and we would like to develop automatic reading compr… (voir plus)ehension systems that can do the same. In this paper, we introduce a task and several models to drive progress towards this goal. In particular, we propose the task of rare entity prediction: given a web document with several entities removed, models are tasked with predicting the correct missing entities conditioned on the document context and the lexical resources. This task is challenging due to the diversity of language styles and the extremely large number of rare entities. We propose two recurrent neural network architectures which make use of external knowledge in the form of entity descriptions. Our experiments show that our hierarchical LSTM model performs significantly better at the rare entity prediction task than those that do not make use of external resources.
2017-09-01
Conference on Empirical Methods in Natural Language Processing (publié)
doi.org
Predicting Success in Goal-Driven Human-Human Dialogues
Michael Noseworthy
Jackie Cheung
Joelle Pineau
In goal-driven dialogue systems, success is often defined based on a structured definition of the goal. This requires that the dialogue syst… (voir plus)em be constrained to handle a specific class of goals and that there be a mechanism to measure success with respect to that goal. However, in many human-human dialogues the diversity of goals makes it infeasible to define success in such a way. To address this scenario, we consider the task of automatically predicting success in goal-driven human-human dialogues using only the information communicated between participants in the form of text. We build a dataset from stackoverflow.com which consists of exchanges between two users in the technical domain where ground-truth success labels are available. We then propose a turn-based hierarchical neural network model that can be used to predict success without requiring a structured goal definition. We show this model outperforms rule-based heuristics and other baselines as it is able to detect patterns over the course of a dialogue and capture notions such as gratitude.
2017-08-01
SIGDIAL Conference (publié)
doi.org
Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support
M. Cardoso
Tal Arbel
G. Carneiro
T. Syeda-Mahmood
J. Tavares
Mehdi Moradi
Andrew P. Bradley
Hayit Greenspan
J. Papa
Anant. Madabhushi
Jacinto C Nascimento
Jaime S. Cardoso
Vasileios Belagiannis
Zhi Lu
Faculdade Engenharia
2017-07-19
ArXiv (prépublication)
doi.org
arxiv.org