Mila is primarily machine learning with focus on deep learning and reinforcement learning
We are always looking forward to push the cutting edge research to new boundaries. Mila is best known for:
- Fundamentals of deep learning: Mila published one of the first papers on deep learning (NIPS’2006), a book (2009) and a review paper (2013), as well as many fundamental contributions regarding representation learning. Ian Goodfellow (former PhD student at MILA), Yoshua Bengio and Aaron Courville published one of the most comprehensive books on deep learning in 2016.
- Autoencoders: Mila first became involved in the deep learning revolution by introducing the stacked denoising autoencoder. Mila has also invented contractive autoencoders and generative stochastic networks. There has been also an on going research on Variational Autoencoders which led to invention of Recurrent Variational Autoencoders and a few other varients at Mila.
- Supervised neural nets: Mila pioneered the use of rectified linear units for feedforward neural nets. Deep rectifier nets have gained widespread popularity in industrial vision and speech recognition. Mila also developed the Attention Mechanism methods, which have gained popularity in a variety of speech and vision applications.
- Generative models: Mila is the birthplace of the spike-and-slab RBM, and also Generative Adversarial Networks (GANs) as one of the best generative models of natural images. Students at Mila study a wide range of topics related to GANs, including better ways to train both the discriminator and the generator.
- Recurrent neural networks: Students at Mila have been studying the problem of “capturing temporal structures” with recurrent nets for a long time. Neural language models were invented at Mila. The birthplace of the RNN-RBM, a very successful model of polyphonic music is also Mila.
Mila is sponsored by industrial partners
- IBM. Mila is doing fundamental research sponsored by IBM on various aspects of deep learning, especially for language-related tasks and computer vision.
- Google wants to build and train efficiently much larger models than what is currently possible and we have introduced the concept of distributed conditional computation and dynamically structured networks to achieve this objective.
- Nuance cares about improvements in learning algorithms for speech and language.
- Facebook cares about improvements in all aspects of deep learning.
- And many more.
There are many advantages to being a student at Mila:
- Vibrant intellectual research environment: at Mila we are all excited to advance the state of artificial intelligence. At any given day you can easily find students discussing all kinds of ambitious research ideas. The professors are very involved and engaged with the students, making the research process very open and interactive.
- Quebecois work/life balance: at Mila we do good research, but we don’t work ourselves to death doing it. The culture here is much more laid back than it is at other labs, especially in the US, that publish at the same conferences.
- Lots of computational resources: University of Montreal has the second largest research budget of any university in Canada, and Mila has a stellar track record of attracting grant money. This means Mila students have access to plenty of fast computers with modern GPUs. The institute’s resources are supplemented with access to shared computer clusters across Quebec and Canada in general.
- Funding for conferences: So long as we have the funding, which we usually do, Mila sends a large delegation to the top machine learning and deep learning conferences like NIPS, ICML, and ICLR. It’s common for Master’s students, second authors, third authors, and even students who haven’t had a paper accepted to get to go to a conference. Professors here strongly encourage attending conferences as an integral part of the life of a researcher. This compares favorably to many other labs where students are discouraged from taking time away from research or where there is rarely funding to send many students.
- Research-based funding: Mila students are generally paid scholarships that are competitive with top US graduate programs. This funding is usually contingent on working on research projects related to one of the lab’s many industrial projects. These projects are often publishable. Most students find this arrangement preferable to some of the arrangements at other universities, where grad student funding is often tied to large amounts of non-research-related work, such as TAing.
- Internships: Mila students commonly do internships at top industrial research groups. Recent examples include:
- Sarath Chandar, Montreal Google Brain (2017)
- Dzmitry Bahdanau, London DeepMind (2017)
- Tim Cooijmans, London DeepMind (2017)
- Tong Che, Berkeley, CA (2017)
- Vincent Dumoulin, London DeepMind (2017)
What are the projects at Mila?
Most projects involve deep learning, though recently the institute has broadened its interest to wider machine learning including optimization. Projects range from natural language understanding to computer vision, through work on general-purpose advances in algorithms.
Why should someone with good math / systems CS skills work on deep learning?
Deep learning is an exciting new research area. It is young enough that a student with good mathematical aptitude can quickly learn most of the central ideas and begin contributing to the field. Recently, deep learning has proven to be a resource-intensive field, with large neural networks powered by cutting edge GPU-based or distributed implementations making great leaps forward in fields like object recognition and speech recognition. Deep learning research presents an unparalleled opportunity for students with good mathematical skills and / or good CS implementation skills to really make a big impact.
Which professors are looking for students and what kind of projects do they want to work on?
Please visit the professor’s webpage to learn about their research interests.
Mila has five CS faculty members:
one faculty at the Engineering school, specializing in machine learning for computer vision:
and two faculty in the business school (HEC) specializing in decision and recommendation algorithms:
And several associate members (who are available for co-supervision):
You can apply through Mila admission to start working with them.
Do I need to speak French?
French is the official language of Quebec, and it is a good idea to learn French if you choose to live here, in order to experience to the fullest extent all that the city has to offer.
However, Montreal is one of the most bilingual locations on Earth, and many current students arrived at the institute with little or no proficiency in French.
The relative popularity of English and French at the institute fluctuates over time. Currently, many students in the institute originate from non-French-speaking countries such as the US, China, Turkey and Iran, so everyone in the institute speaks English reasonably well, whereas French is a common language to a smaller subset.
Courses at the University of Montreal are taught in French, but it is possible to take some of your required courses in English at McGill or Concordia, which are a short bus ride away from University of Montreal campus. Even at University of Montreal, it is preferred that students who speak French poorly turn in their assignments and exams written in English and can ask the professor for an English version of the questions (which the professor almost always accepts to provide). Undergraduate students must pass a French proficiency exam but Master’s and PhD students need not do so.
Will I need to pay extra fees as a foreign student?
By default, foreign students do need to pay extra fees, but Mila students are usually able to get these fees covered by one of the institute’s funding sources, or obtain an exemption. You will be notified whether you will have to pay foreign student fees before you need to formally accept or reject an offer of admission. Keep in mind that one needs to stay in good standing in terms of grades (and publications also help) to continue receiving the tuition fee scholarship provided by the university.
Isn’t it cold in Montreal?
It’s true that Montreal is cold in the winter (and, in fact, very hot in the summer), but the city is well-adapted to function in spite of the weather. Montreal’s snow removal is incredibly fast and efficient. You will need to get a good coat, hat, gloves, and boots, but most foreign students don’t find that they mind the cold too much, which is not very different from other cities such as Boston or Toronto.
The climate also provides certain advantages. There is an ice skating lake within walking distance of the institute and good skiing (both downhill and cross country) within reasonable driving distance of Montreal. It’s easy to go sledding in one of Montreal’s many parks — just grab a glossy cardboard box out of a recycling bin on the street and tear off one of its sides.
What have alumni of the institute done with their degrees?
- Pascal Vincent obtained his PhD at Mila and is still a member of MILA. He is now an Assistant Professor.
- Hugo Larochelle obtained his PhD at Mila and is the head of Google Brain in Montreal.
- Dumitru Erhan obtained his PhD at Mila and is now a software engineer with Google’s visual search team in Venice, CA near Los Angeles.
- James Bergstra obtained his PhD at Mila and is now a post-doctoral research scientist at the University of Waterloo.
- Nicolas Le Roux obtained his PhD at Mila and is now Scientific Program Manager at Criteo.
- Nicolas Chapados obtained his PhD at Mila and now works at ApSTAT, a company he co-founded with other members of Mila.
- Olivier Delalleau obtained his PhD at Mila and now works at Ubisoft in Montreal.
- Philippe Hamel obtained his PhD at Mila and now works at Google in Mountain View, CA.
- Guillaume Desjardins obtained his PhD at Mila and is now a Senior Researcher at Google Deep Mind in London.
- Razvan Pascanu recently obtained his PhD at Mila and is now a Senior Researcher at Google Deep Mind in London.
- Ian Goodfellow recently obtained his PhD at Mila and is now a Research Scientist with Google’s deep learning infrastructure team in Mountain View, CA.