Portrait de Marc-Alexandre Côté

Marc-Alexandre Côté

Membre industriel associé
Microsoft Research
Sujets de recherche
Agent basé sur un LLM
Apprentissage par renforcement
Apprentissage profond
Raisonnement
Site web
Google Scholar
LinkedIn

Publications

Z-Forcing: Training Stochastic Recurrent Networks
Anirudh Goyal
Alessandro Sordoni
Marc-Alexandre Côté
Nan Rosemary Ke
Yoshua Bengio
Many efforts have been devoted to training generative latent variable models with autoregressive decoders, such as recurrent neural networks… (voir plus) (RNN). Stochastic recurrent models have been successful in capturing the variability observed in natural sequential data such as speech. We unify successful ideas from recently proposed architectures into a stochastic recurrent model: each step in the sequence is associated with a latent variable that is used to condition the recurrent dynamics for future steps. Training is performed with amortized variational inference where the approximate posterior is augmented with a RNN that runs backward through the sequence. In addition to maximizing the variational lower bound, we ease training of the latent variables by adding an auxiliary cost which forces them to reconstruct the state of the backward recurrent network. This provides the latent variables with a task-independent objective that enhances the performance of the overall model. We found this strategy to perform better than alternative approaches such as KL annealing. Although being conceptually simple, our model achieves state-of-the-art results on standard speech benchmarks such as TIMIT and Blizzard and competitive performance on sequential MNIST. Finally, we apply our model to language modeling on the IMDB dataset where the auxiliary cost helps in learning interpretable latent variables. Source Code: https://github.com/anirudh9119/zforcing_nips17
2016-12-31
Advances in Neural Information Processing Systems 30 (NIPS 2017) (publié)
doi.org
arxiv.org
Theano: A Python framework for fast computation of mathematical expressions
Rami Al-Rfou
Guillaume Alain
Amjad Almahairi
Christof Angermueller
Dzmitry Bahdanau
Nicolas Ballas
Frédéric Bastien
Justin Bayer
Anatoly Belikov
Alexander Belopolsky
Yoshua Bengio
Arnaud Bergeron
James Bergstra
Valentin Bisson
Josh Bleecher Snyder
Nicolas Bouchard
Nicolas Boulanger-Lewandowski
Xavier Bouthillier
Alexandre De Brébisson
Olivier Breuleux … (voir 92 de plus)
Pierre-Luc Carrier
Kyunghyun Cho
Jan Chorowski
Paul Christiano
Tim Cooijmans
Marc-Alexandre Côté
Myriam Côté
Aaron Courville
Yann N. Dauphin
Olivier Delalleau
Julien Demouth
Guillaume Desjardins
Sander Dieleman
Laurent Dinh
Mélanie Ducoffe
Vincent Dumoulin
Samira Ebrahimi Kahou
Dumitru Erhan
Ziye Fan
Orhan Firat
Mathieu Germain
Xavier Glorot
Ian Goodfellow
Matt Graham
Caglar Gulçehre
Philippe Hamel
Iban Harlouchet
Jean-philippe Heng
Balázs Hidasi
Sina Honari
Arjun Jain
Sébastien Jean
Kai Jia
Mikhail Korobov
Vivek Kulkarni
Alex Lamb
Pascal Lamblin
Eric Larsen
César Laurent
Sean Lee
Simon Lefrancois
Simon Lemieux
Nicholas Léonard
Zhouhan Lin
Jesse A. Livezey
Cory Lorenz
Jeremiah Lowin
Qianli Ma
Pierre-Antoine Manzagol
Olivier Mastropietro
Robert T. McGibbon
Roland Memisevic
Bart Van Merriënboer
Vincent Michalski
Mehdi Mirza
Alberto Orlandi
Christopher Pal
Razvan Pascanu
Mohammad Pezeshki
Colin Raffel
Daniel Renshaw
Matthew Rocklin
Adriana Romero
Markus Roth
Peter Sadowski
John Salvatier
François Savard
Jan Schlüter
John Schulman
Gabriel Schwartz
Iulian Vlad Serban
Dmitriy Serdyuk
Samira Shabanian
Etienne Simon
Sigurd Spieckermann
S. Ramana Subramanyam
Jakub Sygnowski
Jeremie Tanguay
Gijs van Tulder
Joseph Turian
Sebastian Urban
Pascal Vincent
Francesco Visin
Harm de Vries
David Warde-Farley
Dustin J. Webb
Matthew Willson
Kelvin Xu
Lijun Xue
Li Yao
Saizheng Zhang
Ying Zhang
Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficie… (voir plus)ntly. Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements. Theano is being actively and continuously developed since 2008, multiple frameworks have been built on top of it and it has been used to produce many state-of-the-art machine learning models. The present article is structured as follows. Section I provides an overview of the Theano software and its community. Section II presents the principal features of Theano and how to use them, and compares them with other similar projects. Section III focuses on recently-introduced functionalities and improvements. Section IV compares the performance of Theano against Torch7 and TensorFlow on several machine learning models. Section V discusses current limitations of Theano and potential ways of improving it.
2015-12-31
arXiv (prépublication)
doi.org
arxiv.org