Guillaume Rabusseau

farzaneh.heidari@mila.quebec

Beheshteh Toloueirakhshan

PhD - Université de Montréal

rakhshab@mila.quebec

Farzaneh Heidari

PhD - Université de Montréal

Co-supervisor :

Jian Tang

Julia Gastinger

Collaborating Alumni - University of Mannheim

Co-supervisor :

Reihaneh Rabbany

julia.gastinger@mila.quebec

Jun Dai

Postdoctorate - Université de Montréal

jun.dai@mila.quebec

marawan.gamal@mila.quebec

Marawan Gamal

PhD - Université de Montréal

michael.rizvi-martel@mila.quebec

Maude Lizaire

PhD - Université de Montréal

lizairem@mila.quebec

Michael Rizvi-Martel

Master's Research - Université de Montréal

Website

Shirzadkhani Razieh Shirzadkhani

Omar Chikar

Master's Research - Université de Montréal

omar.chikhar@mila.quebec

Collaborating researcher

Co-supervisor :

Reihaneh Rabbany

razieh.shirzadkhani@mila.quebec

soroush.omranpour@mila.quebec

Shenyang Huang

PhD - McGill University

Principal supervisor :

Master's Research - McGill University

Principal supervisor :

Reihaneh Rabbany

Publications

Representation of Reinforcement Learning Policies in Reproducing Kernel Hilbert Spaces.

Bogdan Mazoure

Thang Doan

Tianyu Li

We propose a general framework for policy representation for reinforcement learning tasks. This framework involves finding a low-dimensional… (see more) embedding of the policy on a reproducing kernel Hilbert space (RKHS). The usage of RKHS based methods allows us to derive strong theoretical guarantees on the expected return of the reconstructed policy. Such guarantees are typically lacking in black-box models, but are very desirable in tasks requiring stability. We conduct several experiments on classic RL domains. The results confirm that the policies can be robustly embedded in a low-dimensional space while the embedded policy incurs almost no decrease in return.

2020-02-07

(published)

www.semanticscholar.org

Tensorized Random Projections

Beheshteh T. Rakhshan

2020-01-01

AISTATS (published)

proceedings.mlr.press

Efficient Planning under Partial Observability with Unnormalized Q Functions and Spectral Learning

Tianyu Li

Bogdan Mazoure

Learning and planning in partially-observable domains is one of the most difficult problems in reinforcement learning. Traditional methods c… (see more)onsider these two problems as independent, resulting in a classical two-stage paradigm: first learn the environment dynamics and then plan accordingly. This approach, however, disconnects the two problems and can consequently lead to algorithms that are sample inefficient and time consuming. In this paper, we propose a novel algorithm that combines learning and planning together. Our algorithm is closely related to the spectral learning algorithm for predicitive state representations and offers appealing theoretical guarantees and time complexity. We empirically show on two domains that our approach is more sample and time efficient compared to classical methods.

2019-11-01

ArXiv (preprint)

Neural Architecture Search for Class-incremental Learning

Shenyang Huang

Vincent Francois-Lavet

In class-incremental learning, a model learns continuously from a sequential data stream in which new classes occur. Existing methods often … (see more)rely on static architectures that are manually crafted. These methods can be prone to capacity saturation because a neural network's ability to generalize to new concepts is limited by its fixed capacity. To understand how to expand a continual learner, we focus on the neural architecture design problem in the context of class-incremental learning: at each time step, the learner must optimize its performance on all classes observed so far by selecting the most competitive neural architecture. To tackle this problem, we propose Continual Neural Architecture Search (CNAS): an autoML approach that takes advantage of the sequential nature of class-incremental learning to efficiently and adaptively identify strong architectures in a continual learning setting. We employ a task network to perform the classification task and a reinforcement learning agent as the meta-controller for architecture search. In addition, we apply network transformations to transfer weights from previous learning step and to reduce the size of the architecture search space, thus saving a large amount of computational resources. We evaluate CNAS on the CIFAR-100 dataset under varied incremental learning scenarios with limited computational power (1 GPU). Experimental results demonstrate that CNAS outperforms architectures that are optimized for the entire dataset. In addition, CNAS is at least an order of magnitude more efficient than naively using existing autoML methods.

2019-09-14

ArXiv (preprint)

Recognizable series on graphs and hypergraphs

Raphael Bailly

François Denis

2019-09-01

Journal of Computer and System Sciences (published)

doi.org

Connecting Weighted Automata and Recurrent Neural Networks through Spectral Learning

Tianyu Li

In this paper, we unravel a fundamental connection between weighted finite automata~(WFAs) and second-order recurrent neural networks~(2-RNN… (see more)s): in the case of sequences of discrete symbols, WFAs and 2-RNNs with linear activation functions are expressively equivalent. Motivated by this result, we build upon a recent extension of the spectral learning algorithm to vector-valued WFAs and propose the first provable learning algorithm for linear 2-RNNs defined over sequences of continuous input vectors. This algorithm relies on estimating low rank sub-blocks of the so-called Hankel tensor, from which the parameters of a linear 2-RNN can be provably recovered. The performances of the proposed method are assessed in a simulation study.

2019-04-11

Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics (published)

proceedings.mlr.press

Clustering-Oriented Representation Learning with Attractive-Repulsive Loss

Kian Kenyon-Dean

Andre Cianflone

Lucas Caccia

Jackie Cheung

The standard loss function used to train neural network classifiers, categorical cross-entropy (CCE), seeks to maximize accuracy on the trai… (see more)ning data; building useful representations is not a necessary byproduct of this objective. In this work, we propose clustering-oriented representation learning (COREL) as an alternative to CCE in the context of a generalized attractive-repulsive loss framework. COREL has the consequence of building latent representations that collectively exhibit the quality of natural clustering within the latent space of the final hidden layer, according to a predefined similarity function. Despite being simple to implement, COREL variants outperform or perform equivalently to CCE in a variety of scenarios, including image and news article classification using both feed-forward and convolutional neural networks. Analysis of the latent spaces created with different similarity functions facilitates insights on the different use cases COREL variants can satisfy, where the Cosine-COREL variant makes a consistently clusterable latent space, while Gaussian-COREL consistently obtains better classification accuracy than CCE.

2018-12-18

ArXiv (preprint)

Connecting Weighted Automata and Recurrent Neural Networks through Spectral Learning

Tianyu Li

2018-07-04

ArXiv (preprint)

Minimization of Graph Weighted Models over Circular Strings

2018-04-16

Foundations of Software Science and Computation Structure (published)

doi.org

Nonlinear Weighted Finite Automata

Tianyu Li

Weighted finite automata (WFA) can expressively model functions defined over strings but are inherently linear models. Given the recent succ… (see more)esses of nonlinear models in machine learning, it is natural to wonder whether extending WFA to the nonlinear setting would be beneficial. In this paper, we propose a novel model of neural network based nonlinear WFA model (NL-WFA) along with a learning algorithm. Our learning algorithm is inspired by the spectral learning algorithm for WFA and relies on a nonlinear decomposition of the so-called Hankel matrix, by means of an auto-encoder network. The expressive power of NL-WFA and the proposed learning algorithm are assessed on both synthetic and real world data, showing that NL-WFA can lead to smaller model sizes and infer complex grammatical structures from data.

2018-03-31

International Conference on Artificial Intelligence and Statistics (published)

dblp.uni-trier.de

Sequential Coordination of Deep Models for Learning Visual Arithmetic

Eric E Crawford

Joelle Pineau

Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in… (see more) one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive. Consider a visual arithmetic task, where the goal is to carry out simple arithmetical algorithms on digits presented under natural conditions (e.g. hand-written, placed randomly). We propose a two-tiered architecture for tackling this problem. The lower tier consists of a heterogeneous collection of information processing modules, which can include pre-trained deep neural networks for locating and extracting characters from the image, as well as modules performing symbolic transformations on the representations extracted by perception. The higher tier consists of a controller, trained using reinforcement learning, which coordinates the modules in order to solve the high-level task. For instance, the controller may learn in what contexts to execute the perceptual networks and what symbolic transformations to apply to their outputs. The resulting model is able to solve a variety of tasks in the visual arithmetic domain, and has several advantages over standard, architecturally homogeneous feedforward networks including improved sample efficiency.

2018-02-15

ArXiv (preprint)

Learning Graph Weighted Models on Pictures

Philip Amortila

Graph Weighted Models (GWMs) have recently been proposed as a natural generalization of weighted automata over strings and trees to arbitrar… (see more)y families of labeled graphs (and hypergraphs). A GWM generically associates a labeled graph with a tensor network and computes a value by successive contractions directed by its edges. In this paper, we consider the problem of learning GWMs defined over the graph family of pictures (or 2-dimensional words). As a proof of concept, we consider regression and classification tasks over the simple Bars & Stripes and Shifting Bits picture languages and provide an experimental study investigating whether these languages can be learned in the form of a GWM from positive and negative examples using gradient-based methods. Our results suggest that this is indeed possible and that investigating the use of gradient-based methods to learn picture series and functions computed by GWMs over other families of graphs could be a fruitful direction.

2018-01-01

ICGI (published)

proceedings.mlr.press