Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies
Posted on16 Sep 2019
Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish during training,...Read More
Recurrent Batch Normalization
Posted on30 Mar 2016
We propose a reparameterization of LSTM that brings the benefits of batch normalization to recurrent neural networks. Whereas previous works only apply...Read More
Regularizing RNNs by Stabilizing Activations
Posted on24 Nov 2015
We stabilize the activations of Recurrent Neural Networks (RNNs) by penalizing the squared distance between successive hidden states’ norms. This penalty term...Read More
A Recurrent Latent Variable Model for Sequential Data
Posted on07 Jun 2015
In this paper, we explore the inclusion of random variables into the dynamic hidden state of a recurrent neural network (RNN) by...Read More