NLP in the era of generative AI, cognitive sciences, and societal transformation
Join us at Mila in October for a three-day workshop to explore the transformative potential of language technologies and their implications for society.
This program is designed to provide decision-makers, policymakers and professional working in policy with a foundational understanding of AI technology.
We use cookies to analyze the browsing and usage of our website and to personalize your experience. You can disable these technologies at any time, but this may limit certain functionalities of the site. Read our Privacy Policy for more information.
Setting cookies
You can enable and disable the types of cookies you wish to accept. However certain choices you make could affect the services offered on our sites (e.g. suggestions, personalised ads, etc.).
Essential cookies
These cookies are necessary for the operation of the site and cannot be deactivated. (Still active)
Analytics cookies
Do you accept the use of cookies to measure the audience of our sites?
Multimedia Player
Do you accept the use of cookies to display and allow you to watch the video content hosted by our partners (YouTube, etc.)?
Publications
Association between extreme precipitation, drinking water and acute gastrointestinal illness in the Great Lakes
Domain modelling transforms domain problem descriptions written in natural language (NL) into analyzable and concise domain models (class di… (see more)agrams) during requirements analysis or the early stages of design in software development. Since the practice of domain modelling requires time in addition to modelling skills and experience, several approaches have been proposed to automate or semi-automate the construction of domain models from problem descriptions expressed in NL. Despite the existing work on domain model extraction, some significant challenges remain unaddressed: (i) the extracted domain models are not accurate enough to be used directly or with minor modifications in software development, (ii) existing approaches do not facilitate the tracing of the rationale behind the modelling decisions taken by the model extractor, and (iii) existing approaches do not provide interactive interfaces to update the extracted domain models. Therefore, in this paper, we introduce a domain modelling bot called DoMoBOT, explain its architecture, and implement it in the form of a web-based prototype tool. The bot automatically extracts a domain model from a problem description written in NL with an accuracy higher than existing approaches. Furthermore, the bot enables modellers to update a part of the extracted domain model and in response the bot re-configures the other parts of the domain model pro-actively. To improve the accuracy of extracted domain models, we combine the techniques of Natural Language Processing and Machine Learning. Finally, we evaluate the accuracy of the extracted domain models.
2020-10-26
Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings (published)
Background: The Canadian epidemics of COVID-19 exhibit distinct early trajectories, with Quebec bearing a very high initial burden. The sema… (see more)ine de relache, or March break, took place two weeks earlier in Quebec as compared to the rest of Canada. This event may have played a role in the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). We aimed to examine the role of case importation in the early transmission dynamics of SARS-CoV-2 in Quebec. Methods: Using detailed surveillance data, we developed and calibrated a deterministic SEIR-type compartmental model of SARS-CoV-2 transmission. We explored the impact of altering the number of imported cases on hospitalizations. Specifically, we investigated scenarios without case importation after March break, and as scenarios where cases were imported with the same frequency/timing as neighboring Ontario. Results: A total of 1,544 and 1,150 returning travelers were laboratory-confirmed in Quebec and Ontario, respectively (with symptoms onset before 2020-03-25). The cumulative number of hospitalizations could have been reduced by 55% (95% credible interval [95%CrI]: 51-59%) had no cases been imported after Quebec's March break. However, had Quebec experienced Ontario's number of imported cases, cumulative hospitalizations would have only been reduced by 12% (95%CrI: 8-16%). Interpretation: Our results suggest that case importation played an important role in the early spread of COVID-19 in Quebec. Yet, heavy importation of SARS-CoV-2 in early March could be insufficient to resolve interprovincial heterogeneities in cumulative hospitalizations. The importance of other factors -public health preparedness, responses, and capacity- should be investigated.
2020-10-25
International Journal of Infectious Diseases (published)
The role of case importation in explaining differences in early SARS-CoV-2 transmission dynamics in Canada—A mathematical modeling study of surveillance data
Syntax is fundamental to our thinking about language. Although neural networks are very successful in many tasks, they do not explicitly mod… (see more)el syntactic structure. Failing to capture the structure of inputs could lead to generalization problems and over-parametrization. In the present work, we propose a new syntax-aware language model: Syntactic Ordered Memory (SOM). The model explicitly models the structure with a one-step look-ahead parser and maintains the conditional probability setting of the standard language model. Experiments show that SOM can achieve strong results in language modeling and syntactic generalization tests, while using fewer parameters then other models.
Quantum Tensor Networks, Stochastic Processes, and Weighted Automata
Siddarth Srinivasan
Sandesh M. Adhikary
Jacob Miller
Guillaume Rabusseau
Byron Boots
Modeling joint probability distributions over sequences has been studied from many perspectives. The physics community developed matrix prod… (see more)uct states, a tensor-train decomposition for probabilistic modeling, motivated by the need to tractably model many-body systems. But similar models have also been studied in the stochastic processes and weighted automata literature, with little work on how these bodies of work relate to each other. We address this gap by showing how stationary or uniform versions of popular quantum tensor network models have equivalent representations in the stochastic processes and weighted automata literature, in the limit of infinitely long sequences. We demonstrate several equivalence results between models used in these three communities: (i) uniform variants of matrix product states, Born machines and locally purified states from the quantum tensor networks literature, (ii) predictive state representations, hidden Markov models, norm-observable operator models and hidden quantum Markov models from the stochastic process literature,and (iii) stochastic weighted automata, probabilistic automata and quadratic automata from the formal languages literature. Such connections may open the door for results and methods developed in one area to be applied in another.
Feed-forward neural networks consist of a sequence of layers, in which each layer performs some processing on the information from the previ… (see more)ous layer. A downside to this approach is that each layer (or module, as multiple modules can operate in parallel) is tasked with processing the entire hidden state, rather than a particular part of the state which is most relevant for that module. Methods which only operate on a small number of input variables are an essential part of most programming languages, and they allow for improved modularity and code re-usability. Our proposed method, Neural Function Modules (NFM), aims to introduce the same structural capability into deep learning. Most of the work in the context of feed-forward networks combining top-down and bottom-up feedback is limited to classification problems. The key contribution of our work is to combine attention, sparsity, top-down and bottom-up feedback, in a flexible algorithm which, as we show, improves the results in standard classification, out-of-domain generalization, generative modeling, and learning representations in the context of reinforcement learning.
Abstract The bulk of social neuroscience takes a ‘stimulus-brain’ approach, typically comparing brain responses to different types of so… (see more)cial stimuli, but most of the time in the absence of direct social interaction. Over the last two decades, a growing number of researchers have adopted a ‘brain-to-brain’ approach, exploring similarities between brain patterns across participants as a novel way to gain insight into the social brain. This methodological shift has facilitated the introduction of naturalistic social stimuli into the study design (e.g. movies) and, crucially, has spurred the development of new tools to directly study social interaction, both in controlled experimental settings and in more ecologically valid environments. Specifically, ‘hyperscanning’ setups, which allow the simultaneous recording of brain activity from two or more individuals during social tasks, has gained popularity in recent years. However, currently, there is no agreed-upon approach to carry out such ‘inter-brain connectivity analysis’, resulting in a scattered landscape of analysis techniques. To accommodate a growing demand to standardize analysis approaches in this fast-growing research field, we have developed Hyperscanning Python Pipeline, a comprehensive and easy open-source software package that allows (social) neuroscientists to carry-out and to interpret inter-brain connectivity analyses.
2020-10-08
Social Cognitive and Affective Neuroscience (published)
A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix
Thang Doan
Mehdi Abbana Bennani
Bogdan Mazoure
Guillaume Rabusseau
Pierre Alquier
Continual learning (CL) is a setting in which an agent has to learn from an incoming stream of data during its entire lifetime. Although maj… (see more)or advances have been made in the field, one recurring problem which remains unsolved is that of Catastrophic Forgetting (CF). While the issue has been extensively studied empirically, little attention has been paid from a theoretical angle. In this paper, we show that the impact of CF increases as two tasks increasingly align. We introduce a measure of task similarity called the NTK overlap matrix which is at the core of CF. We analyze common projected gradient algorithms and demonstrate how they mitigate forgetting. Then, we propose a variant of Orthogonal Gradient Descent (OGD) which leverages structure of the data through Principal Component Analysis (PCA). Experiments support our theoretical findings and show how our method reduces CF on classical CL datasets.