Publications

Vision-Language Pretraining: Current Trends and the Future
Aishwarya Agrawal
Damien Teney
Aida Nematzadeh
In the last few years, there has been an increased interest in building multimodal (vision-language) models that are pretrained on larger bu… (voir plus)t noisier datasets where the two modalities (e.g., image and text) loosely correspond to each other (e.g., Lu et al., 2019; Radford et al., 2021). Given a task (such as visual question answering), these models are then often fine-tuned on task-specific supervised datasets. (e.g., Lu et al., 2019; Chen et al.,2020; Tan and Bansal, 2019; Li et al., 2020a,b). In addition to the larger pretraining datasets, the transformer architecture (Vaswani et al., 2017) and in particular self-attention applied to two modalities are responsible for the impressive performance of the recent pretrained models on downstream tasks (Hendricks et al., 2021). In this tutorial, we focus on recent vision-language pretraining paradigms. Our goal is to first provide the background on image–language datasets, benchmarks, and modeling innovations before the multimodal pretraining area. Next we discuss the different family of models used for vision-language pretraining, highlighting their strengths and shortcomings. Finally, we discuss the limits of vision-language pretraining through statistical learning, and the need for alternative approaches such as causal representation learning.
2022-01-01
Annual Meeting of the Association for Computational Linguistics (published)
doi.org
Vision-Language Pretraining: Current Trends and the Future
Aishwarya Agrawal
Damien Teney
Aida Nematzadeh
In the last few years, there has been an increased interest in building multimodal (vision-language) models that are pretrained on larger bu… (voir plus)t noisier datasets where the two modalities (e.g., image and text) loosely correspond to each other (e.g., Lu et al., 2019; Radford et al., 2021). Given a task (such as visual question answering), these models are then often fine-tuned on task-specific supervised datasets. (e.g., Lu et al., 2019; Chen et al.,2020; Tan and Bansal, 2019; Li et al., 2020a,b). In addition to the larger pretraining datasets, the transformer architecture (Vaswani et al., 2017) and in particular self-attention applied to two modalities are responsible for the impressive performance of the recent pretrained models on downstream tasks (Hendricks et al., 2021). In this tutorial, we focus on recent vision-language pretraining paradigms. Our goal is to first provide the background on image–language datasets, benchmarks, and modeling innovations before the multimodal pretraining area. Next we discuss the different family of models used for vision-language pretraining, highlighting their strengths and shortcomings. Finally, we discuss the limits of vision-language pretraining through statistical learning, and the need for alternative approaches such as causal representation learning.
2022-01-01
Annual Meeting of the Association for Computational Linguistics (publié)
doi.org
Washing The Unwashable : On The (Im)possibility of Fairwashing Detection
A. Shamsabadi
Mohammad Yaghini
Natalie Dullerud
Sierra Calanda Wyllie
Ulrich Aivodji
Aisha Alaagib
Sébastien Gambs
Nicolas Papernot
What does it mean to be an AI Ethicist: An ontology of existing roles
Shalaleh Rismani
AJung Moon
With the increasing adoption of Artificial Intelligence systems (AIS) in various application and the growing efforts to regulate such system… (voir plus)s, a new set of occupations has emerged in the industry. This new set of roles take different titles and hold varying responsibilities. However, the individuals in these roles are tasked with interpreting and operationalizing best practices for developing ethical and safe AI systems. We will broadly refer to this new set of occupations as AI ethicists and recognize that they often hold a specific role in the intersection of technology development, business needs, and societal implications. In this work, we examine what it means to be an AI ethicist in the industry and propose an ontology of existing roles under this broad title along with their required competencies. We create this ontology by examining the job postings for such roles over the past two years and conduct expert interviews with fourteen individuals who currently hold such a role in the industry. The proposed ontology will inform executives and leaders who are looking to build responsible AI teams and provide educators the necessary information for creating new learning objectives and curriculum.
2022-01-01
arXiv.org (prépublication)
doi.org
What does it mean to be an AI Ethicist: An ontology of existing roles
Shalaleh Rismani
AJung Moon
With the increasing adoption of Artificial Intelligence systems (AIS) in various application and the growing efforts to regulate such system… (voir plus)s, a new set of occupations has emerged in the industry. This new set of roles take different titles and hold varying responsibilities. However, the individuals in these roles are tasked with interpreting and operationalizing best practices for developing ethical and safe AI systems. We will broadly refer to this new set of occupations as AI ethicists and recognize that they often hold a specific role in the intersection of technology development, business needs, and societal implications. In this work, we examine what it means to be an AI ethicist in the industry and propose an ontology of existing roles under this broad title along with their required competencies. We create this ontology by examining the job postings for such roles over the past two years and conduct expert interviews with fourteen individuals who currently hold such a role in the industry. The proposed ontology will inform executives and leaders who are looking to build responsible AI teams and provide educators the necessary information for creating new learning objectives and curriculum.
2022-01-01
arXiv.org (preprint)
doi.org
Machine learning application development: practitioners’ insights
Md. Saidur Rahman
Foutse Khomh
Alaleh Hamidi
Jinghui Cheng
Giuliano Antoniol
Hironori Washizaki
2021-12-31
ArXiv (preprint)
doi.org
arxiv.org
Heterogeneous Crowd Simulation Using Parametric Reinforcement Learning
Kaidong Hu
Brandon Haworth
Glen Berseth
Vladimir Pavlovic
Petros Faloutsos
Mubbasir Kapadia
Agent-based synthetic crowd simulation affords the cost-effective large-scale simulation and animation of interacting digital humans. Model-… (voir plus)based approaches have successfully generated a plethora of simulators with a variety of foundations. However, prior approaches have been based on statically defined models predicated on simplifying assumptions, limited video-based datasets, or homogeneous policies. Recent works have applied reinforcement learning to learn policies for navigation. However, these approaches may learn static homogeneous rules, are typically limited in their generalization to trained scenarios, and limited in their usability in synthetic crowd domains. In this article, we present a multi-agent reinforcement learning-based approach that learns a parametric predictive collision avoidance and steering policy. We show that training over a parameter space produces a flexible model across crowd configurations. That is, our goal-conditioned approach learns a parametric policy that affords heterogeneous synthetic crowds. We propose a model-free approach without centralization of internal agent information, control signals, or agent communication. The model is extensively evaluated. The results show policy generalization across unseen scenarios, agent parameters, and out-of-distribution parameterizations. The learned model has comparable computational performance to traditional methods. Qualitatively the model produces both expected (laminar flow, shuffling, bottleneck) and unexpected (side-stepping) emergent qualitative behaviours, and quantitatively the approach is performant across measures of movement quality.
2021-12-29
IEEE Transactions on Visualization and Computer Graphics (publié)
doi.org
Heterogeneous Crowd Simulation Using Parametric Reinforcement Learning
Kaidong Hu
Michael Brandon Haworth
Glen Berseth
Vladimir Pavlovic
Petros Faloutsos
Mubbasir. T. Kapadia
Agent-based synthetic crowd simulation affords the cost-effective large-scale simulation and animation of interacting digital humans. Model-… (voir plus)based approaches have successfully generated a plethora of simulators with a variety of foundations. However, prior approaches have been based on statically defined models predicated on simplifying assumptions, limited video-based datasets, or homogeneous policies. Recent works have applied reinforcement learning to learn policies for navigation. However, these approaches may learn static homogeneous rules, are typically limited in their generalization to trained scenarios, and limited in their usability in synthetic crowd domains. In this article, we present a multi-agent reinforcement learning-based approach that learns a parametric predictive collision avoidance and steering policy. We show that training over a parameter space produces a flexible model across crowd configurations. That is, our goal-conditioned approach learns a parametric policy that affords heterogeneous synthetic crowds. We propose a model-free approach without centralization of internal agent information, control signals, or agent communication. The model is extensively evaluated. The results show policy generalization across unseen scenarios, agent parameters, and out-of-distribution parameterizations. The learned model has comparable computational performance to traditional methods. Qualitatively the model produces both expected (laminar flow, shuffling, bottleneck) and unexpected (side-stepping) emergent qualitative behaviours, and quantitatively the approach is performant across measures of movement quality.
2021-12-29
IEEE Transactions on Visualization and Computer Graphics (published)
doi.org
Single Allocation Hub Location with Heterogeneous Economies of Scale
Borzou Rostami
Masoud Chitsaz
Okan Arslan
Gilbert Laporte
Andrea Lodi
The economies of scale in hub location is usually modeled by a constant parameter, which captures the benefits companies obtain through cons… (voir plus)olidation. In their article “Single allocation hub location with heterogeneous economies of scale,” Rostami et al. relax this assumption and consider hub-hub connection costs as piecewise linear functions of the flow amounts. This spoils the triangular inequality property of the distance matrix, making the classical flow-based model invalid and further complicates the problem. The authors tackle the challenge by building a mixed-integer quadratically constrained program and by developing a methodology based on constructing Lagrangian function, linear dual functions, and specialized polynomial-time algorithms to generate enhanced cuts. The developed method offers a new strategy in Benders-type decomposition through relaxing a set of complicating constraints in subproblems when such relaxation is tight. The results confirm the efficacy of the solution methods in solving large-scale problem instances.
2021-12-28
Operational Research (published)
doi.org
COVID-19 Seroprevalence in Canada Modelling Waning and Boosting COVID-19 Immunity in Canada a Canadian Immunization Research Network Study
David W. Dick
Lauren Childs
Zhilan Feng
Jing Li
Gergely Röst
David Buckeridge
Nick H. Ogden
Jane Heffernan
2021-12-23
Vaccines (publié)
doi.org
Fall 2021 Resurgence and COVID-19 Seroprevalence in Canada: Modelling waning and boosting COVID-19 immunity in Canada, A Canadian Immunization Research Network Study
David W. Dick
Lauren Childs
Zhilan Feng
Jing Li
Gergely Röst
David Buckeridge
Nick H. Ogden
Jane Heffernan
2021-12-23
Vaccines (publié)
doi.org
Generative Models of Brain Dynamics -- A review
Mahta Ramezanian Panahi
Germán Abrevaya
Jean-Christophe Gagnon-Audet
Vikram Voleti
Irina Rish
Guillaume Dumas
The principled design and discovery of biologically- and physically-informed models of neuronal dynamics has been advancing since the mid-tw… (voir plus)entieth century. Recent developments in artificial intelligence (AI) have accelerated this progress. This review article gives a high-level overview of the approaches across different scales of organization and levels of abstraction. The studies covered in this paper include fundamental models in computational neuroscience, nonlinear dynamics, data-driven methods, as well as emergent practices. While not all of these models span the intersection of neuroscience, AI, and system dynamics, all of them do or can work in tandem as generative models, which, as we argue, provide superior properties for the analysis of neuroscientific data. We discuss the limitations and unique dynamical traits of brain data and the complementary need for hypothesis- and data-driven modeling. By way of conclusion, we present several hybrid generative models from recent literature in scientific machine learning, which can be efficiently deployed to yield interpretable models of neural dynamics.
2021-12-22
ArXiv (prépublication)
arxiv.org
arxiv.org