Hugo Larochelle

Membre industriel principal

Professeur associé, Université de Montréal, Département d'informatique et de recherche opérationnelle

Chercheur scientifique

Sujets de recherche

Apprentissage profond

Biographie

Je suis chercheur et professeur associé à l'Université de Montréal. Mes recherches se spécialisent dans l'étude et le développement de méthodes d'apprentissage profond.

Étudiants actuels

Sangnie Bhardwaj

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Guillaume Lajoie

Mélisande Astrid Crystal Teng

Doctorat - UdeM

Superviseur⋅e principal⋅e :

Yoshua Bengio

Site web

Github

Hattie Zhou

Doctorat - UdeM

Co-superviseur⋅e :

Aaron Courville

Publications

Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations

David Scott Krueger

Tegan Maharaj

J'anos Kram'ar

Mohammad Pezeshki

Nicolas Ballas

Nan Rosemary Ke

Anirudh Goyal

We propose zoneout, a novel method for regularizing RNNs. At each timestep, zoneout stochastically forces some hidden units to maintain thei… (voir plus)r previous values. Like dropout, zoneout uses random noise to train a pseudo-ensemble, improving generalization. But by preserving instead of dropping hidden units, gradient information and state information are more readily propagated through time, as in feedforward stochastic depth networks. We perform an empirical investigation of various RNN regularizers, and find that zoneout gives significant performance improvements across tasks. We achieve competitive results with relatively simple models in character- and word-level language modelling on the Penn Treebank and Text8 datasets, and combining with recurrent batch normalization yields state-of-the-art results on permuted sequential MNIST.

2016-06-03

ArXiv (prépublication)

arxiv.org