NLP in the era of generative AI, cognitive sciences, and societal transformation
Join us at Mila in October for a three-day workshop to explore the transformative potential of language technologies and their implications for society.
This program is designed to provide decision-makers, policymakers and professional working in policy with a foundational understanding of AI technology.
We use cookies to analyze the browsing and usage of our website and to personalize your experience. You can disable these technologies at any time, but this may limit certain functionalities of the site. Read our Privacy Policy for more information.
Setting cookies
You can enable and disable the types of cookies you wish to accept. However certain choices you make could affect the services offered on our sites (e.g. suggestions, personalised ads, etc.).
Essential cookies
These cookies are necessary for the operation of the site and cannot be deactivated. (Still active)
Analytics cookies
Do you accept the use of cookies to measure the audience of our sites?
Multimedia Player
Do you accept the use of cookies to display and allow you to watch the video content hosted by our partners (YouTube, etc.)?
Publications
A Unified, Scalable Framework for Neural Population Decoding
Our ability to use deep learning approaches to decipher neural activity would likely benefit from greater scale, in terms of both the model … (see more)size and the datasets. However, the integration of many neural recordings into one unified model is challenging, as each recording contains the activity of different neurons from different individual animals. In this paper, we introduce a training framework and architecture designed to model the population dynamics of neural activity across diverse, large-scale neural recordings. Our method first tokenizes individual spikes within the dataset to build an efficient representation of neural events that captures the fine temporal structure of neural activity. We then employ cross-attention and a PerceiverIO backbone to further construct a latent tokenization of neural population activities. Utilizing this architecture and training framework, we construct a large-scale multi-session model trained on large datasets from seven nonhuman primates, spanning over 158 different sessions of recording from over 27,373 neural units and over 100 hours of recordings. In a number of different tasks, we demonstrate that our pretrained model can be rapidly adapted to new, unseen sessions with unspecified neuron correspondence, enabling few-shot performance with minimal labels. This work presents a powerful new approach for building deep learning tools to analyze neural data and stakes out a clear path to training at scale for neural decoding models.
Reinforcement learning (RL) algorithms face two distinct challenges: learning effective representations of past and present observations, an… (see more)d determining how actions influence future returns. Both challenges involve modeling long-term dependencies. The Transformer architecture has been very successful to solve problems that involve long-term dependencies, including in the RL domain. However, the underlying reason for the strong performance of Transformer-based RL methods remains unclear: is it because they learn effective memory, or because they perform effective credit assignment? After introducing formal definitions of memory length and credit assignment length, we design simple configurable tasks to measure these distinct quantities. Our empirical results reveal that Transformers can enhance the memory capability of RL algorithms, scaling up to tasks that require memorizing observations
Phylogenetic diversity (PD)—the evolutionary history of a set of species—is conceptually linked to the maintenance of yet-to-be-discover… (see more)ed benefits from biodiversity or “option value.” We used global phylogenetic and utilization data for birds to test the PD option value link, under the assumption that the performance of sets of PD-maximizing species at capturing known benefits is analogous to selecting the same species at a point in human history before these benefits were realized. PD performed better than random at capturing utilized bird species across 60% of tests, with performance linked to the phylogenetic dispersion and prevalence of each utilization category. Prioritizing threatened species for conservation by the PD they encapsulate performs comparably to prioritizing by their functional distinctiveness. However, species selected by each metric show low overlap, indicating that we should conserve both components of biodiversity to effectively conserve a variety of uses. Our findings provide empirical support for the link between evolutionary history and benefits for future generations.
Background Non-pharmaceutical interventions (NPIs) and vaccines have been widely used to manage the COVID-19 pandemic. However, uncertainty … (see more)persists regarding the effectiveness of these interventions due to data quality issues, methodological challenges, and differing contextual factors. Accurate estimation of their effects is crucial for future epidemic preparedness. Methods To address this, we developed a population-based mechanistic model that includes the impact of NPIs and vaccines on SARS-CoV-2 transmission and hospitalization rates. Our statistical approach estimated all parameters in one step, accurately propagating uncertainty. We fitted the model to comprehensive epidemiological data in France from March 2020 to October 2021. With the same model, we simulated scenarios of vaccine rollout. Results The first lockdown was the most effective, reducing transmission by 84% (95% confidence interval (CI) 83-85). Subsequent lockdowns had diminished effectiveness (reduction of 74% (69-77) and 11% (9-18), respectively). A 6pm curfew was more effective than one at 8 pm (68% (66-69) vs. 48% (45-49) reduction), while school closures reduced transmission by 15% (12-18). In a scenario without vaccines before November 2021, we predicted 159,000 or 194% (95% prediction interval (PI) 74-424) more deaths and 1,488,000 or 340% (136-689) more hospitalizations. If a vaccine had been available after 100 days, over 71,000 deaths (16,507-204,249) and 384,000 (88,579-1,020,386) hospitalizations could have been averted. Conclusion Our results highlight the substantial impact of NPIs, including lockdowns and curfews, in controlling the COVID-19 pandemic. We also demonstrate the value of the 100 days objective of the CEPI initiative for vaccine availability.
Background: Dementia is a neurodegenerative disease resulting in the loss of cognitive and psychological functions. Artificial intelligence … (see more)(AI) may help in detection and screening of dementia; however, little is known in this area. Objectives: The objective of this study was to identify and evaluate AI interventions for detection of dementia using motion data. Method: The review followed the framework proposed by O’Malley’s and Joanna Briggs Institute methodological guidance for scoping reviews. We adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist for reporting the results. An information specialist performed a comprehensive search from the date of inception until November 2020, in five bibliographic databases: MEDLINE, EMBASE, Web of Science Core Collection, CINAHL, and IEEE Xplore. We included studies aimed at the deployment and testing or implementation of AI interventions using motion data for the detection of dementia among a diverse population, encompassing varying age, sex, gender, economic backgrounds, and ethnicity, extending to their health care providers across multiple health care settings. Studies were excluded if they focused on Parkinson’s or Huntington’s disease. Two independent reviewers screened the abstracts, titles, and then read the full-texts. Disagreements were resolved by consensus, and if this was not possible, the opinion of a third reviewer was sought. The reference lists of included studies were also screened. Results: After removing duplicates, 2,632 articles were obtained. After title and abstract screening and full-text screening, 839 articles were considered for categorization. The authors categorized the papers into six categories, and data extraction and synthesis was performed on 20 included papers from the motion tracking data category. The included studies assessed cognitive performance (n = 5, 25%); screened dementia and cognitive decline (n = 8, 40%); investigated visual behaviours (n = 4, 20%); and analyzed motor behaviors (n = 3, 15%). Conclusions: We presented evidence of AI systems being employed in the detection of dementia, showcasing the promising potential of motion tracking within this domain. Although some progress has been made in this field recently, there remain notable research gaps that require further exploration and investigation. Future endeavors need to compare AI interventions using motion data with traditional screening methods or other tech-enabled dementia detection mechanisms. Besides, future works should aim at understanding how gender and sex, and ethnic and cultural sensitivity can contribute to refining AI interventions, ensuring they are accessible, equitable, and beneficial across all society.
2023-09-13
Dementia and Geriatric Cognitive Disorders EXTRA (published)