Eugene Belilovsky

pietro.cagnasso@mila.quebec

Xiao Huang

Maîtrise recherche - Concordia

Co-superviseur⋅e :

xiaolong.huang@mila.quebec

Paul Janson

Doctorat - Concordia

Co-superviseur⋅e :

paul.janson@mila.quebec

Gwen Legate

Doctorat - Concordia

Co-superviseur⋅e :

gwendolyne.legate@mila.quebec

Paria Mehrbod

Maîtrise recherche - Concordia

Co-superviseur⋅e :

paria.mehrbod@mila.quebec

Abhinav Moudgil

Doctorat - Concordia

abhinav.moudgil@mila.quebec

Adel Nabli

Doctorat - Concordia

adel.nabli@mila.quebec

Geraldin Nanfack

Postdoctorat - Concordia

Co-superviseur⋅e :

geraldin.nanfack@mila.quebec

Albert Orozco Camacho

Doctorat - Concordia

Co-superviseur⋅e :

Doctorat - Concordia

Co-superviseur⋅e :

vaibhav.singh@mila.quebec

Benjamin Therien

Doctorat - UdeM

Superviseur⋅e principal⋅e :

benjamin.therien@mila.quebec

Stagiaire de recherche - Concordia

junior-cedric.tonga@mila.quebec

Zhang Yanlei

Collaborateur·rice alumni - UdeM

Superviseur⋅e principal⋅e :

yanlei.zhang@mila.quebec

amirhossein.zamani@mila.quebec

Amirhossein Zamani

Doctorat - Concordia

Co-superviseur⋅e :

Blake Richards

Publications

Graph Density-Aware Losses for Novel Compositions in Scene Graph Generation

Boris Knyazev

Harm de Vries

Cătălina Cangea

Graham W. Taylor

Aaron Courville

Eugene Belilovsky

Scene graph generation (SGG) aims to predict graph-structured descriptions of input images, in the form of objects and relationships between… (voir plus) them. This task is becoming increasingly useful for progress at the interface of vision and language. Here, it is important - yet challenging - to perform well on novel (zero-shot) or rare (few-shot) compositions of objects and relationships. In this paper, we identify two key issues that limit such generalization. Firstly, we show that the standard loss used in this task is unintentionally a function of scene graph density. This leads to the neglect of individual edges in large sparse graphs during training, even though these contain diverse few-shot examples that are important for generalization. Secondly, the frequency of relationships can create a strong bias in this task, such that a blind model predicting the most frequent relationship achieves good performance. Consequently, some state-of-the-art models exploit this bias to improve results. We show that such models can suffer the most in their ability to generalize to rare compositions, evaluating two different models on the Visual Genome dataset and its more recent, improved version, GQA. To address these issues, we introduce a density-normalized edge loss, which provides more than a two-fold improvement in certain generalization metrics. Compared to other works in this direction, our enhancements require only a few lines of code and no added computational cost. We also highlight the difficulty of accurately evaluating models using existing metrics, especially on zero/few shots, and introduce a novel weighted metric.

2019-12-31

Proceedings of the British Machine Vision Conference 2020 (publié)

doi.org

Online Continual Learning with Maximally Interfered Retrieval

Lucas Caccia

Massimo Caccia

Tinne Tuytelaars

Continual learning, the setting where a learning agent is faced with a never ending stream of data, continues to be a great challenge for mo… (voir plus)dern machine learning systems. In particular the online or "single-pass through the data" setting has gained attention recently as a natural setting that is difficult to tackle. Methods based on replay, either generative or from a stored memory, have been shown to be effective approaches for continual learning, matching or exceeding the state of the art in a number of standard benchmarks. These approaches typically rely on randomly selecting samples from the replay memory or from a generative model, which is suboptimal. In this work, we consider a controlled sampling of memories for replay. We retrieve the samples which are most interfered, i.e. whose prediction will be most negatively impacted by the foreseen parameters update. We show a formulation for this sampling criterion in both the generative replay and the experience replay setting, producing consistent gains in performance and greatly reduced forgetting. We release an implementation of our method at this https URL.

2019-08-10

ArXiv (prépublication)

VideoNavQA: Bridging the Gap between Visual and Embodied Question Answering

Cătălina Cangea

Eugene Belilovsky

Pietro Lio

Aaron Courville

Embodied Question Answering (EQA) is a recently proposed task, where an agent is placed in a rich 3D environment and must act based solely o… (voir plus)n its egocentric input to answer a given question. The desired outcome is that the agent learns to combine capabilities such as scene understanding, navigation and language understanding in order to perform complex reasoning in the visual world. However, initial advancements combining standard vision and language methods with imitation and reinforcement learning algorithms have shown EQA might be too complex and challenging for these techniques. In order to investigate the feasibility of EQA-type tasks, we build the VideoNavQA dataset that contains pairs of questions and videos generated in the House3D environment. The goal of this dataset is to assess question-answering performance from nearly-ideal navigation paths, while considering a much more complete variety of questions than current instantiations of the EQA task. We investigate several models, adapted from popular VQA methods, on this new benchmark. This establishes an initial understanding of how well VQA-style methods can perform within this novel EQA paradigm.

2018-12-31

ViGIL@NeurIPS (publié)

doi.org

Blindfold Baselines for Embodied QA

We explore blindfold (question-only) baselines for Embodied Question Answering. The EmbodiedQA task requires an agent to answer a question b… (voir plus)y intelligently navigating in a simulated environment, gathering necessary visual information only through first-person vision before finally answering. Consequently, a blindfold baseline which ignores the environment and visual information is a degenerate solution, yet we show through our experiments on the EQAv1 dataset that a simple question-only baseline achieves state-of-the-art results on the EmbodiedQA task in all cases except when the agent is spawned extremely close to the object.

2018-11-11

ArXiv (prépublication)

doi.org