Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Posted on10/06/2014
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent... Read More
On the Number of Linear Regions of Deep Neural Networks
Posted on08/02/2014
We study the complexity of functions computable by deep feedforward neural networks with piecewise linear activations in terms of the symmetries and... Read More
Bounding the Test Log-Likelihood of Generative Model
Posted on24/12/2013
Several interesting generative learning algorithms involve a complex probability distribution over many random variables, involving intractable normalization constants or latent variable normalization.... Read More
On the number of response regions of deep feed forward networks with piece-wise linear activations
Posted on20/12/2013
This paper explores the complexity of deep feedforward networks with linear pre-synaptic couplings and rectified linear activations. This is a contribution to... Read More
Stochastic k-Neighborhood Selection for Supervised and Unsupervised Learning
Posted on23/06/2013
Neighborhood Components Analysis (NCA) is a popular method for learning a distance metric to be used within a k-nearest neighbors (kNN) classifier.... Read More
The Toronto Paper Matching System: An automated paper-reviewer assignment system
Posted on23/06/2013
One of the most important tasks of conference organizers is the assignment of papers to reviewers. Reviewers’ assessments of papers is a... Read More
Active Learning for Matching Problems
Posted on23/06/2012
Effective learning of user preferences is critical to easing user burden in various types of matching problems. Equally important is active query... Read More
A Framework for Optimizing Paper Matching
Posted on23/07/2011
At the heart of many scientific conferences is the problem of matching submitted papers to suitable reviewers. Arriving at a good assignment... Read More
Deep Sparse Rectifier Neural Networks
Posted on01/03/2011
While logistic sigmoid neurons are more bi- ologically plausible than hyperbolic tangent neurons, the latter work better for train- ing multi-layer neural... Read More
Understanding the difficulty of training deep feedforward neural networks
Posted on01/03/2010
Whereas before 2006 it appears that deep multi- layer neural networks were not successfully trained, since then several algorithms have been shown... Read More
