Publications

Deep discriminant analysis for task-dependent compact network search
Qing Tian
Tal Arbel
James J. Clark
Most of today's popular deep architectures are hand-engineered for general purpose applications. However, this design procedure usually lead… (see more)s to massive redundant, useless, or even harmful features for specific tasks. Such unnecessarily high complexities render deep nets impractical for many real-world applications, especially those without powerful GPU support. In this paper, we attempt to derive task-dependent compact models from a deep discriminant analysis perspective. We propose an iterative and proactive approach for classification tasks which alternates between (1) a pushing step, with an objective to simultaneously maximize class separation, penalize co-variances, and push deep discriminants into alignment with a compact set of neurons, and (2) a pruning step, which discards less useful or even interfering neurons. Deconvolution is adopted to reverse `unimportant' filters' effects and recover useful contributing sources. A simple network growing strategy based on the basic Inception module is proposed for challenging tasks requiring larger capacity than what the base net can offer. Experiments on the MNIST, CIFAR10, and ImageNet datasets demonstrate our approach's efficacy. On ImageNet, by pushing and pruning our grown Inception-88 model, we achieve better-performing models than smaller deep Inception nets grown, residual nets, and famous compact nets at similar sizes. We also show that our grown deep Inception nets (without hard-coded dimension alignment) can beat residual nets of similar complexities.
2020-09-28
arXiv.org (preprint)
dblp.uni-trier.de
Learning to Summarize Long Texts with Memory Compression and Transfer
Jaehong Park
Jonathan Pilault
Christopher Pal
2020-09-27
ArXiv (preprint)
arxiv.org
arxiv.org
Generating Multiscale Amorphous Molecular Structures Using Deep Learning: A Study in 2D.
Michael Kilgour
Nicolas Gastellu
David Y. T. Hui
Yoshua Bengio
Lena Simine
Amorphous molecular assemblies appear in a vast array of systems: from living cells to chemical plants and from everyday items to new device… (see more)s. The absence of long-range order in amorphous materials implies that precise knowledge of their underlying structures throughout is needed to rationalize and control their properties at the mesoscale. Standard computational simulations suffer from exponentially unfavorable scaling of the required compute with system size. We present a method based on deep learning that leverages the finite range of structural correlations for an autoregressive generation of disordered molecular aggregates up to arbitrary size from small-scale computational or experimental samples. We benchmark performance on self-assembled nanoparticle aggregates and proceed to simulate monolayer amorphous carbon with atomistic resolution. This method bridges the gap between the nanoscale and mesoscale simulations of amorphous molecular systems.
2020-09-23
Journal of Physical Chemistry Letters (published)
doi.org
A learning-based algorithm to quickly compute good primal solutions for Stochastic Integer Programs
Yoshua Bengio
Emma Frejinger
Andrea Lodi
Rahul Patel
Sriram Sankaranarayanan
We propose a novel approach using supervised learning to obtain near-optimal primal solutions for two-stage stochastic integer programming (… (see more)2SIP) problems with constraints in the first and second stages. The goal of the algorithm is to predict a "representative scenario" (RS) for the problem such that, deterministically solving the 2SIP with the random realization equal to the RS, gives a near-optimal solution to the original 2SIP. Predicting an RS, instead of directly predicting a solution ensures first-stage feasibility of the solution. If the problem is known to have complete recourse, second-stage feasibility is also guaranteed. For computational testing, we learn to find an RS for a two-stage stochastic facility location problem with integer variables and linear constraints in both stages and consistently provide near-optimal solutions. Our computing times are very competitive with those of general-purpose integer programming solvers to achieve a similar solution quality.
2020-09-18
Integration of Constraint Programming, Artificial Intelligence, and Operations Research (published)
doi.org
arxiv.org
A normative modelling approach reveals age-atypical cortical thickness in a subgroup of males with autism spectrum disorder
Richard A.I. Bethlehem
Jakob Seidlitz
Rafael Romero-Garcia
Stavros Trakoshis
Guillaume Dumas
Michael V. Lombardo
2020-09-03
Communications Biology (published)
doi.org
Keynote Lecture - Building Knowledge For AI AgentsWith Reinforcement Learning
Doina Precup
Summary form only given, as follows. The complete presentation was not made available for publication as part of the conference proceedings.… (see more) Reinforcement learning allows autonomous agents to learn how to act in a stochastic, unknown environment, with which they can interact. Deep reinforcement learning, in particular, has achieved great success in well-defined application domains, such as Go or chess, in which an agent has to learn how to act and there is a clear success criterion. In this talk, I will focus on the potential role of reinforcement learning as a tool for building knowledge representations in AI agents whose goal is to perform continual learning. I will examine a key concept in reinforcement learning, the value function, and discuss its generalization to support various forms of predictive knowledge. I will also discuss the role of temporally extended actions, and their associated predictive models, in learning procedural knowledge. In order to tame the possible complexity of learning knowledge representations, reinforcement learning agents can use the concepts of intents (ie intended consequences of courses of actions) and affordances (which capture knowlege about where actions can be applied). Finally, I will discuss the challenge of how to evaluate reinforcement learning agents whose goal is not just to control their environment, but also to build knowledge about their world.
2020-09-02
International Conference on Computational Photography (published)
doi.org
Shared Decision Making in Surgery: A Meta-Analysis of Existing Literature
Kacper Niburski
Elena Guadagno
S. A. Rahimi
Dan Poenaru
2020-09-02
The Patient - Patient-Centered Outcomes Research (published)
doi.org
Correction to: Why public health matters today and tomorrow: the role of applied public health research
Lindsay McLaren
Paula Braitstein
David L Buckeridge
Damien Contandriopoulos
Maria I. Creatore
Guy Faulkner
David Hammond
Steven J. Hoffman
Yan Kestens
Scott Leatherdale
Jonathan McGavock
Wendy V. Norman
Candace Nykiforuk
Valéry Ridde
Janet Smylie
The article “Why public health matters today and tomorrow: the role of applied public health research,” written by Lindsay McLaren et al… (see more)., was originally published Online First without Open Access.
2020-09-01
Canadian Journal of Public Health = Revue Canadienne de Santé Publique (published)
doi.org
Learning General Transformations of Data for Out-of-Sample Extensions
Matthew Amodio
David van Dijk
Guy Wolf
Smita Krishnaswamy
While generative models such as GANs have been successful at mapping from noise to specific distributions of data, or more generally from on… (see more)e distribution of data to another, they cannot isolate the transformation that is occurring and apply it to a new distribution not seen in training. Thus, they memorize the domain of the transformation, and cannot generalize the transformation out of sample. To address this, we propose a new neural network called a Neuron Transformation Network (NTNet) that isolates the signal representing the transformation itself from the other signals representing internal distribution variation. This signal can then be removed from a new dataset distributed differently from the original one trained on. We demonstrate the effectiveness of our NTNet on more than a dozen synthetic and biomedical single-cell RNA sequencing datasets, where the NTNet is able to learn the data transformation performed by genetic and drug perturbations on one sample of cells and successfully apply it to another sample of cells to predict treatment outcome.
2020-08-31
Europe PubMed Central (accepted)
doi.org
SENET: A Semantic Web for Supporting Automation of Software Engineering Tasks
Yalin Liu
Jinfeng Lin
Jane Cleland-Huang
Michael Vierhauser
Jin L.C. Guo
Sugandha Lohar
The use of Natural Language (NL) interfaces to allow devices and applications to respond to verbal commands or free-form textual queries is … (see more)becoming increasingly prevalent in our society. To a large extent, their success in interpreting and responding to a request is dependent upon rich underlying ontologies and conceptual models that understand the technical or domain specific vocabulary of diverse users. The effective use of NL interfaces in the Software Engineering (SE) domains requires its own ontology models focusing upon software related terms and concepts. While many SE glossaries exist, they are often incomplete and tend to define the vocabulary for specific sub-fields without capturing associations between terms and phrases. This limits their usefulness for supporting NL-related tasks. In this paper we propose an approach for constructing and evolving a semantic network of software engineering concepts and phrases. Our approach starts with a set of existing SE glossaries, uses the existing glossary terms and explicitly defined associations as a starting point, uses machine learning-based techniques to dynamically identify and document additional associations between terms, leverages the network to interpret NL queries in the SE domain, and finally augments the resulting semantic network with feedback provided by users. We evaluate the viability of our approach within the sub-domain of Agile Software Development, focusing on requirements related queries, and show that the semantic network enhances the ability of an NL interface to correctly interpret and execute user queries.
2020-08-31
International Workshop on Artificial Intelligence for Requirements Engineering (published)
doi.org
On the Effectiveness of Two-Step Learning for Latent-Variable Models
Cem Subakan
Maxime Gasse
Laurent Charlin
Latent-variable generative models offer a principled solution for modeling and sampling from complex probability distributions. Implementing… (see more) a joint training objective with a complex prior, however, can be a tedious task, as one is typically required to derive and code a specific cost function for each new type of prior distribution. In this work, we propose a general framework for learning latent variable generative models in a two-step fashion. In the first step of the framework, we train an autoencoder, and in the second step we fit a prior model on the resulting latent distribution. This two-step approach offers a convenient alternative to joint training, as it allows for a straightforward combination of existing models without the hustle of deriving new cost functions, and the need for coding the joint training objectives. Through a set of experiments, we demonstrate that two-step learning results in performances similar to joint training, and in some cases even results in more accurate modeling.
2020-08-31
International Workshop on Machine Learning for Signal Processing (published)
doi.org
Traceability Network Analysis: A Case Study of Links in Issue Tracking Systems
Alexander Nicholson
Deeksha M. Arya
Jin L.C. Guo
Traceability links between software artifacts serve as an invaluable resource for reasoning about software products and their development pr… (see more)ocess. Most conventional methods for capturing traceability are based on pair-wise artifact relations such as trace matrices or navigable links between two directly related artifacts. However, this limited view of trace links ignores the propagating effect of artifact connections as well as the trace link properties at a project level. In this work, we propose the use of network structures to provide another perspective from which reasoning on a collective of trace events is possible. We explore various network analysis techniques in the issue tracking system of sixty-six open source projects. Our observation reveals two salient properties of the traceability network, i.e. scale free and triadic closure. These properties provide a strong indication of the applicability of network analysis tools and can be used to identify and examine important "hub" issues. As a stepping stone, these properties can further support project status analysis and link type prediction. As a proof-of-concept, we demonstrate the effectiveness of applying the triadic closure property to link type prediction.
2020-08-31
International Workshop on Artificial Intelligence for Requirements Engineering (published)
doi.org