Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies
Posted on16/09/2019
Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish during training,... Read More
Recurrent Batch Normalization
Posted on30/03/2016
We propose a reparameterization of LSTM that brings the benefits of batch normalization to recurrent neural networks. Whereas previous works only apply... Read More
Regularizing RNNs by Stabilizing Activations
Posted on24/11/2015
We stabilize the activations of Recurrent Neural Networks (RNNs) by penalizing the squared distance between successive hidden states’ norms. This penalty term... Read More
A Recurrent Latent Variable Model for Sequential Data
Posted on07/06/2015
In this paper, we explore the inclusion of random variables into the dynamic hidden state of a recurrent neural network (RNN) by... Read More