Portrait of Yoshua Bengio

Yoshua Bengio

Core Academic Member
Canada CIFAR AI Chair
Full Professor, Université de Montréal, Department of Computer Science and Operations Research Department
Scientific Director, Leadership Team
Research Topics
Causality
Computational Neuroscience
Deep Learning
Generative Models
Graph Neural Networks
Machine Learning Theory
Medical Machine Learning
Molecular Modeling
Natural Language Processing
Probabilistic Models
Reasoning
Recurrent Neural Networks
Reinforcement Learning
Representation Learning
Website
Google Scholar

Biography

*For media requests, please write to medias@mila.quebec.

For more information please contact Marie-Josée Beauchamp, Administrative Assistant at marie-josee.beauchamp@mila.quebec.

Yoshua Bengio is recognized worldwide as a leading expert in AI. He is most known for his pioneering work in deep learning, which earned him the 2018 A.M. Turing Award, “the Nobel Prize of computing,” with Geoffrey Hinton and Yann LeCun.

Bengio is a full professor at Université de Montréal, and the founder and scientific director of Mila – Quebec Artificial Intelligence Institute. He is also a senior fellow at CIFAR and co-directs its Learning in Machines & Brains program, serves as scientific director of IVADO, and holds a Canada CIFAR AI Chair.

In 2019, Bengio was awarded the prestigious Killam Prize and in 2022, he was the most cited computer scientist in the world by h-index. He is a Fellow of the Royal Society of London, Fellow of the Royal Society of Canada, Knight of the Legion of Honor of France and Officer of the Order of Canada. In 2023, he was appointed to the UN’s Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology.

Concerned about the social impact of AI, Bengio helped draft the Montréal Declaration for the Responsible Development of Artificial Intelligence and continues to raise awareness about the importance of mitigating the potentially catastrophic risks associated with future AI systems.

Current Students

Jamal Abou Haibeh
Research Intern - McGill University
Github
Google Scholar
Mohammed Abukalam
Collaborating Alumni - Université de Montréal
Github
Salwane Agassoussi
Research Intern - Université de Montréal
agassoussisalwane2@gmail.com
Rim Assouel
PhD - Université de Montréal
Ayoub Atanane
Collaborating Alumni - Université du Québec à Rimouski
Github
Stefan Bauer
Independent visiting researcher
Co-supervisor :
Guillaume Lajoie
Google Scholar
Paul Bertin
PhD - Université de Montréal
Ghait Boukachab
Collaborating Alumni - UQAR
Github
Oussama Boussif
PhD - Université de Montréal
Website
Github
Google Scholar
Shahana Chatterjee
Collaborating researcher - N/A
Principal supervisor :
David Rolnick
Xiaoyin Chen
PhD - Université de Montréal
Sanghyeok Choi
Collaborating researcher - KAIST
Website
Github
Google Scholar
Tristan Deleu
PhD - Université de Montréal
Website
Github
Google Scholar
Aniket Didolkar
PhD - Université de Montréal
Website
Github
Google Scholar
Zakarya Elmimouni
Collaborating Alumni - Université de Montréal
Github
Eric Elmoznino
PhD - Université de Montréal
Co-supervisor :
Guillaume Lajoie
Website
Github
Google Scholar
Akram Erraqabi
PhD - Université de Montréal
Léna Nehale Ezzine
PhD - Université de Montréal
Jean-Pierre Falet
PhD - Université de Montréal
Co-supervisor :
Guillaume Lajoie
Website
Github
Google Scholar
Leo Feng
PhD - Université de Montréal
leo.feng@mila.quebec
Website
Google Scholar
Damiano Fornasiere
Research Intern - Barcelona University
Piotr Gainski
Research Intern - Université de Montréal
Github
Ivan Grega
Research Intern - Université de Montréal
Pietro Greiner
Research Intern
Mohsin Hasan
PhD - Université de Montréal
mohsin.hasan@mila.quebec
Website
Github
Google Scholar
Alex Hernandez-Garcia
Postdoctorate - Université de Montréal
Co-supervisor :
David Rolnick
Website
Github
Google Scholar
Edward Hu
PhD - Université de Montréal
Moksh Jain
PhD - Université de Montréal
moksh.jain@mila.quebec
Website
Github
Google Scholar
Thomas Jiralerspong
Master's Research - Université de Montréal
Co-supervisor :
Doina Precup
Website
Github
Google Scholar
Younesse Kaddar
Collaborating Alumni - Université de Montréal
Website
Github
Minsu Kim
Collaborating researcher - Université de Montréal
Website
Github
Google Scholar
Michał Koziarski
Collaborating Alumni - Université de Montréal
Seanie Lee
Collaborating Alumni - Université de Montréal
Website
Github
Google Scholar
Chenghao Liu
Collaborating Alumni
Website
Github
Google Scholar
Zhen Liu
PhD - Université de Montréal
Principal supervisor :
Liam Paull
Matt MacDermott
Collaborating Alumni - Imperial College London
Website
Github
Google Scholar
Kanika Madan
PhD - Université de Montréal
Mohammed Mahfoud
Collaborating Alumni - Université de Montréal
Nikolay Malkin
Collaborating Alumni - Université de Montréal
Website
Github
Google Scholar
Cristian Dragos Manta
PhD - Université de Montréal
Co-supervisor :
Dhanya Sridhar
Github
Sören Mindermann
Collaborating researcher - Université de Montréal
Website
Google Scholar
Sarthak Mittal
PhD - Université de Montréal
Principal supervisor :
Guillaume Lajoie
Website
Github
Google Scholar
Jama Mohamud
PhD - Université de Montréal
Principal supervisor :
Mirco Ravanelli
Website
Github
Google Scholar
Cyrus Neary
Postdoctorate - Université de Montréal
Principal supervisor :
Glen Berseth
Website
Github
Google Scholar
Phong Nguyen
Independent visiting researcher - Université de Montréal
Ali Parviz
Collaborating researcher - Ying Wu Coll of Computing
Google Scholar
Lena Podina
PhD - University of Waterloo
Principal supervisor :
David Rolnick
Google Scholar
Nassim Rahaman
Collaborating Alumni - Max-Planck-Institute for Intelligent Systems
Jarrid Rector-Brooks
PhD - Université de Montréal
Danyal REHMAN
Postdoctorate - Université de Montréal
James Requeima
Independent visiting researcher - Université de Montréal
Oli RICHARDSON
Postdoctorate - Université de Montréal
Website
Github
Google Scholar
Camille Rochefort-Boulanger
PhD - Université de Montréal
Principal supervisor :
Julie Hussin
Github
Victor Schmidt
Collaborating Alumni - Université de Montréal
Website
Github
Google Scholar
Luca Scimeca
Postdoctorate - Université de Montréal
Website
Github
Google Scholar
Dragos Secrieru
Master's Research - Université de Montréal
Marcin Sendera
Collaborating Alumni - Université de Montréal
Github
Google Scholar
Dounia Shaaban Kabakibo
Research Intern - Université de Montréal
Github
Google Scholar
Vedant Shah
Master's Research - Université de Montréal
Website
Github
Google Scholar
Divya Sharma
Collaborating Alumni
Website
Github
Marco Stock
Independent visiting researcher - Technical University of Munich
marco.stock@tum.de
Github
Vincent Taboga
Postdoctorate - Polytechnique Montréal
Co-supervisor :
Pierre-Luc Bacon
Github
Google Scholar
Mélisande Astrid Crystal Teng
PhD - Université de Montréal
Co-supervisor :
Hugo Larochelle
Website
Github
Viktor Todosijevic
Collaborating researcher - RWTH Aachen University (Rheinisch-Westfälische Technische Hochschule Aachen)
Principal supervisor :
David Rolnick
Github
Alex Tong
Postdoctorate - Université de Montréal
alexander.tong@mila.quebec
Website
Github
Google Scholar
Donna Vakalis
Postdoctorate - Université de Montréal
Co-supervisor :
David Rolnick
Github
Google Scholar
Sasha Volokhova
PhD - Université de Montréal
Zichao Yan
Collaborating Alumni - Université de Montréal
Omar Younis
Collaborating researcher
Github
Google Scholar
Taeyoung YUN
Collaborating researcher - KAIST
Github
Tianyu Zhang
PhD - Université de Montréal
Website
Github
Google Scholar
Nicole Zhang
PhD - McGill University
Principal supervisor :
Mathieu Blanchette
Google Scholar
Dinghuai Zhang
PhD - Université de Montréal
Principal supervisor :
Aaron Courville
Website
Harry Zhao
PhD - McGill University
Principal supervisor :
Doina Precup
Website
Github
Google Scholar

Publications

Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke
Konrad Żołna
Alessandro Sordoni
Zhouhan Lin
Adam Trischler
Yoshua Bengio
Joelle Pineau
Laurent Charlin
Chris Pal
Recurrent Neural Networks (RNNs) with attention mechanisms have obtained state-of-the-art results for many sequence processing tasks. Most o… (see more)f these models use a simple form of encoder with attention that looks over the entire sequence and assigns a weight to each token independently. We present a mechanism for focusing RNN encoders for sequence modelling tasks which allows them to attend to key parts of the input as needed. We formulate this using a multi-layer conditional sequence encoder that reads in one token at a time and makes a discrete decision on whether the token is relevant to the context or question being asked. The discrete gating mechanism takes in the context embedding and the current hidden state as inputs and controls information flow into the layer above. We train it using policy gradient methods. We evaluate this method on several types of tasks with different attributes. First, we evaluate the method on synthetic tasks which allow us to evaluate the model for its generalization ability and probe the behavior of the gates in more controlled settings. We then evaluate this approach on large scale Question Answering tasks including the challenging MS MARCO and SearchQA tasks. Our models shows consistent improvements for both tasks over prior work and our baselines. It has also shown to generalize significantly better on synthetic tasks as compared to the baselines.
2018-07-03
Proceedings of the 35th International Conference on Machine Learning (published)
proceedings.mlr.press
arxiv.org
Commonsense mining as knowledge base completion? A study on the impact of novelty
Stanisław Jastrzębski
Dzmitry Bahdanau
Seyedarian Hosseini
Michael Noukhovitch
Yoshua Bengio
Jackie Cheung
Commonsense knowledge bases such as ConceptNet represent knowledge in the form of relational triples. Inspired by recent work by Li et al., … (see more)we analyse if knowledge base completion models can be used to mine commonsense knowledge from raw text. We propose novelty of predicted triples with respect to the training set as an important factor in interpreting results. We critically analyse the difficulty of mining novel commonsense knowledge, and show that a simple baseline method that outperforms the previous state of the art on predicting more novel triples.
2018-06-01
Proceedings of the Workshop on Generalization in the Age of Deep Learning (published)
doi.org
arxiv.org
Learning Anonymized Representations with Adversarial Neural Networks
Clément Feutry
Pablo Piantanida
Yoshua Bengio
P. Duhamel
Statistical methods protecting sensitive information or the identity of the data owner have become critical to ensure privacy of individuals… (see more) as well as of organizations. This paper investigates anonymization methods based on representation learning and deep neural networks, and motivated by novel information theoretical bounds. We introduce a novel training objective for simultaneously training a predictor over target variables of interest (the regular labels) while preventing an intermediate representation to be predictive of the private labels. The architecture is based on three sub-networks: one going from input to representation, one from representation to predicted regular labels, and one from representation to predicted private labels. The training procedure aims at learning representations that preserve the relevant part of the information (about regular labels) while dismissing information about the private labels which correspond to the identity of a person. We demonstrate the success of this approach for two distinct classification versus anonymization tasks (handwritten digits and sentiment analysis).
2018-02-26
ArXiv (preprint)
arxiv.org
arxiv.org
Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks
Nan Rosemary Ke
Anirudh Goyal
Olexa Bilaniuk
Jonathan Binas
Laurent Charlin
Chris Pal
Yoshua Bengio
A major drawback of backpropagation through time (BPTT) is the difficulty of learning long-term dependencies, coming from having to propagat… (see more)e credit information backwards through every single step of the forward computation. This makes BPTT both computationally impractical and biologically implausible. For this reason, full backpropagation through time is rarely used on long sequences, and truncated backpropagation through time is used as a heuristic. However, this usually leads to biased estimates of the gradient in which longer term dependencies are ignored. Addressing this issue, we propose an alternative algorithm, Sparse Attentive Backtracking, which might also be related to principles used by brains to learn long-term dependencies. Sparse Attentive Backtracking learns an attention mechanism over the hidden states of the past and selectively backpropagates through paths with high attention weights. This allows the model to learn long term dependencies while only backtracking for a small number of time steps, not just from the recent past but also from attended relevant past states.
2017-11-07
ArXiv (preprint)
arxiv.org
arxiv.org
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Iulian V. Serban
Alessandro Sordoni
Ryan Lowe
Laurent Charlin
Joelle Pineau
Aaron Courville
Yoshua Bengio
Sequential data often possesses hierarchical structures with complex dependencies between sub-sequences, such as found between the utterance… (see more)s in a dialogue. To model these dependencies in a generative framework, we propose a neural network-based generative architecture, with stochastic latent variables that span a variable number of time steps. We apply the proposed model to the task of dialogue response generation and compare it with other recent neural-network architectures. We evaluate the model performance through a human evaluation study. The experiments demonstrate that our model improves upon recently proposed models and that the latent variables facilitate both the generation of meaningful, long and diverse responses and maintaining dialogue state.
2017-02-12
Proceedings of the AAAI Conference on Artificial Intelligence (published)
doi.org
arxiv.org
Diet Networks: Thin Parameters for Fat Genomics
Adriana Romero Soriano
pierre luc carrier
Akram Erraqabi
Tristan Sylvain
Alex Auvolat
Etienne Dejoie
Marc-André Legault
Marie-Pierre Dubé
Julie Hussin
Yoshua Bengio
Learning tasks such as those involving genomic data often poses a serious challenge: the number of input features can be orders of magnitude… (see more) larger than the number of training examples, making it difficult to avoid overfitting, even when using the known regularization techniques. We focus here on tasks in which the input is a description of the genetic variation specific to a patient, the single nucleotide polymorphisms (SNPs), yielding millions of ternary inputs. Improving the ability of deep learning to handle such datasets could have an important impact in medical research, more specifically in precision medicine, where high-dimensional data regarding a particular patient is used to make predictions of interest. Even though the amount of data for such tasks is increasing, this mismatch between the number of examples and the number of inputs remains a concern. Naive implementations of classifier neural networks involve a huge number of free parameters in their first layer (number of input features times number of hidden units): each input feature is associated with as many parameters as there are hidden units. We propose a novel neural network parametrization which considerably reduces the number of free parameters. It is based on the idea that we can first learn or provide a distributed representation for each input feature (e.g. for each position in the genome where variations are observed in data), and then learn (with another neural network called the parameter prediction network) how to map a feature's distributed representation (based on the feature's identity not its value) to the vector of parameters specific to that feature in the classifier neural network (the weights which link the value of the feature to each of the hidden units). This approach views the problem of producing the parameters associated with each feature as a multi-task learning problem. We show experimentally on a population stratification task of interest to medical studies that the proposed approach can significantly reduce both the number of parameters and the error rate of the classifier.
2017-01-01
ICLR.cc/2017/conference (poster)
openreview.net
openreview.net
Diet Networks: Thin Parameters for Fat Genomics
Adriana Romero Soriano
pierre luc carrier
Akram Erraqabi
Tristan Sylvain
Alex Auvolat
Etienne Dejoie
Marc-André Legault
Marie-Pierre Dubé
Julie Hussin
Yoshua Bengio
Learning tasks such as those involving genomic data often poses a serious challenge: the number of input features can be orders of magnitude… (see more) larger than the number of training examples, making it difficult to avoid overfitting, even when using the known regularization techniques. We focus here on tasks in which the input is a description of the genetic variation specific to a patient, the single nucleotide polymorphisms (SNPs), yielding millions of ternary inputs. Improving the ability of deep learning to handle such datasets could have an important impact in medical research, more specifically in precision medicine, where high-dimensional data regarding a particular patient is used to make predictions of interest. Even though the amount of data for such tasks is increasing, this mismatch between the number of examples and the number of inputs remains a concern. Naive implementations of classifier neural networks involve a huge number of free parameters in their first layer (number of input features times number of hidden units): each input feature is associated with as many parameters as there are hidden units. We propose a novel neural network parametrization which considerably reduces the number of free parameters. It is based on the idea that we can first learn or provide a distributed representation for each input feature (e.g. for each position in the genome where variations are observed in data), and then learn (with another neural network called the parameter prediction network) how to map a feature's distributed representation (based on the feature's identity not its value) to the vector of parameters specific to that feature in the classifier neural network (the weights which link the value of the feature to each of the hidden units). This approach views the problem of producing the parameters associated with each feature as a multi-task learning problem. We show experimentally on a population stratification task of interest to medical studies that the proposed approach can significantly reduce both the number of parameters and the error rate of the classifier.
2017-01-01
ICLR.cc/2017/conference (poster)
openreview.net
openreview.net
Diet Networks: Thin Parameters for Fat Genomic
Adriana Romero Soriano
pierre luc carrier
Akram Erraqabi
Tristan Sylvain
Alex Auvolat
Etienne Dejoie
Marc-andr'e Legault
M. Dubé
Julie Hussin
Yoshua Bengio
Learning tasks such as those involving genomic data often poses a serious challenge: the number of input features can be orders of magnitude… (see more) larger than the number of training examples, making it difficult to avoid overfitting, even when using the known regularization techniques. We focus here on tasks in which the input is a description of the genetic variation specific to a patient, the single nucleotide polymorphisms (SNPs), yielding millions of ternary inputs. Improving the ability of deep learning to handle such datasets could have an important impact in precision medicine, where high-dimensional data regarding a particular patient is used to make predictions of interest. Even though the amount of data for such tasks is increasing, this mismatch between the number of examples and the number of inputs remains a concern. Naive implementations of classifier neural networks involve a huge number of free parameters in their first layer: each input feature is associated with as many parameters as there are hidden units. We propose a novel neural network parametrization which considerably reduces the number of free parameters. It is based on the idea that we can first learn or provide a distributed representation for each input feature (e.g. for each position in the genome where variations are observed), and then learn (with another neural network called the parameter prediction network) how to map a feature's distributed representation to the vector of parameters specific to that feature in the classifier neural network (the weights which link the value of the feature to each of the hidden units). We show experimentally on a population stratification task of interest to medical studies that the proposed approach can significantly reduce both the number of parameters and the error rate of the classifier.
2016-11-04
ArXiv (preprint)
arxiv.org
arxiv.org
HeMIS: Hetero-Modal Image Segmentation
Mohammad Havaei
Nicolas Guizard
Nicolas Chapados
Yoshua Bengio
2016-10-02
Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016 (published)
doi.org
arxiv.org
HeMIS: Hetero-Modal Image Segmentation
Mohammad Havaei
Nicolas Guizard
Nicolas Chapados
Yoshua Bengio
2016-07-18
ArXiv (preprint)
doi.org
arxiv.org
Theano: A Python framework for fast computation of mathematical expressions
Rami Al-rfou'
Guillaume Alain
Amjad Almahairi
Christof Angermüller
Dzmitry Bahdanau
Nicolas Ballas
Frédéric Bastien
Justin S. Bayer
A. Belikov
A. Belopolsky
Yoshua Bengio
Arnaud Bergeron
James Bergstra
Valentin Bisson
Josh Bleecher Snyder
Nicolas Bouchard
Nicolas Boulanger-Lewandowski
Xavier Bouthillier
Alexandre De Brébisson
Olivier Breuleux … (see 92 more)
pierre luc carrier
Kyunghyun Cho
Jan Chorowski
Paul F. Christiano
Tim Cooijmans
Marc-Alexandre Côté
Myriam Côté
Aaron Courville
Yann Dauphin
Olivier Delalleau
Julien Demouth
Guillaume Desjardins
Sander Dieleman
Laurent Dinh
M'elanie Ducoffe
Vincent Dumoulin
Samira Ebrahimi Kahou
Dumitru Erhan
Ziye Fan
Orhan Firat
Mathieu Germain
Xavier Glorot
Ian G Goodfellow
Matthew Graham
Caglar Gulcehre
Philippe Hamel
Iban Harlouchet
Jean-philippe Heng
Balázs Hidasi
Sina Honari
Arjun Jain
Sébastien Jean
Kai Jia
Mikhail V. Korobov
Vivek Kulkarni
Alex Lamb
Pascal Lamblin
Eric Larsen
César Laurent
S. Lee
Simon-mark Lefrancois
Simon Lemieux
Nicholas Léonard
Zhouhan Lin
J. Livezey
Cory R. Lorenz
Jeremiah L. Lowin
Qianli M. Ma
Pierre-Antoine Manzagol
Olivier Mastropietro
R. McGibbon
Roland Memisevic
Bart van Merriënboer
Vincent Michalski
Mehdi Mirza
Alberto Orlandi
Chris Pal
Razvan Pascanu
Mohammad Pezeshki
Colin Raffel
Daniel Renshaw
Matthew David Rocklin
Adriana Romero Soriano
Markus Dr. Roth
Peter Sadowski
John Salvatier
Francois Savard
Jan Schlüter
John D. Schulman
Gabriel Schwartz
Iulian V. Serban
Dmitriy Serdyuk
Samira Shabanian
Etienne Simon
Sigurd Spieckermann
S. Subramanyam
Jakub Sygnowski
Jérémie Tanguay
Gijs van Tulder
Joseph Turian
Sebastian Urban
Pascal Vincent
Francesco Visin
Harm de Vries
David Warde-Farley
Dustin J. Webb
M. Willson
Kelvin Xu
Lijun Xue
Li Yao
Saizheng Zhang
Ying Zhang
Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficie… (see more)ntly. Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements. Theano is being actively and continuously developed since 2008, multiple frameworks have been built on top of it and it has been used to produce many state-of-the-art machine learning models. The present article is structured as follows. Section I provides an overview of the Theano software and its community. Section II presents the principal features of Theano and how to use them, and compares them with other similar projects. Section III focuses on recently-introduced functionalities and improvements. Section IV compares the performance of Theano against Torch7 and TensorFlow on several machine learning models. Section V discusses current limitations of Theano and potential ways of improving it.
2016-05-09
ArXiv (preprint)
arxiv.org
arxiv.org
Theano: A Python framework for fast computation of mathematical expressions
Rami Al-rfou'
Guillaume Alain
Amjad Almahairi
Christof Angermüller
Dzmitry Bahdanau
Nicolas Ballas
Frédéric Bastien
Justin S. Bayer
A. Belikov
A. Belopolsky
Yoshua Bengio
Arnaud Bergeron
J. Bergstra
Valentin Bisson
Josh Bleecher Snyder
Nicolas Bouchard
Nicolas Boulanger-Lewandowski
Xavier Bouthillier
Alexandre De Brébisson
Olivier Breuleux … (see 92 more)
pierre luc carrier
Kyunghyun Cho
Jan Chorowski
Paul F. Christiano
Tim Cooijmans
Marc-Alexandre Côté
Myriam Côté
Aaron Courville
Yann Dauphin
Olivier Delalleau
Julien Demouth
Guillaume Desjardins
Sander Dieleman
Laurent Dinh
M'elanie Ducoffe
Vincent Dumoulin
Samira Ebrahimi Kahou
Dumitru Erhan
Ziye Fan
Orhan Firat
Mathieu Germain
Xavier Glorot
Ian J. Goodfellow
Matthew Graham
Caglar Gulcehre
Philippe Hamel
Iban Harlouchet
Jean-philippe Heng
Balázs Hidasi
Sina Honari
Arjun Jain
S'ebastien Jean
Kai Jia
Mikhail V. Korobov
Vivek Kulkarni
Alex Lamb
Pascal Lamblin
Eric P. Larsen
César Laurent
S. Lee
Simon-mark Lefrancois
Simon Lemieux
Nicholas Léonard
Zhouhan Lin
J. Livezey
Cory R. Lorenz
Jeremiah L. Lowin
Qianli M. Ma
Pierre-Antoine Manzagol
Olivier Mastropietro
R. McGibbon
Roland Memisevic
Bart van Merriënboer
Vincent Michalski
Mehdi Mirza
Alberto Orlandi
Chris Pal
Razvan Pascanu
Mohammad Pezeshki
Colin Raffel
Daniel Renshaw
Matthew David Rocklin
Adriana Romero Soriano
Markus Dr. Roth
Peter Sadowski
John Salvatier
Francois Savard
Jan Schlüter
John D. Schulman
Gabriel Schwartz
Iulian V. Serban
Dmitriy Serdyuk
Samira Shabanian
Etienne Simon
Sigurd Spieckermann
S. Subramanyam
Jakub Sygnowski
Jérémie Tanguay
Gijs van Tulder
Joseph P. Turian
Sebastian Urban
Pascal Vincent
Francesco Visin
Harm de Vries
David Warde-Farley
Dustin J. Webb
M. Willson
Kelvin Xu
Lijun Xue
Li Yao
Saizheng Zhang
Ying Zhang
Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficie… (see more)ntly. Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements. Theano is being actively and continuously developed since 2008, multiple frameworks have been built on top of it and it has been used to produce many state-of-the-art machine learning models. The present article is structured as follows. Section I provides an overview of the Theano software and its community. Section II presents the principal features of Theano and how to use them, and compares them with other similar projects. Section III focuses on recently-introduced functionalities and improvements. Section IV compares the performance of Theano against Torch7 and TensorFlow on several machine learning models. Section V discusses current limitations of Theano and potential ways of improving it.
2016-05-09
ArXiv (preprint)
arxiv.org
arxiv.org