Publications

Sufficient Conditions for Offline Reactivation in Recurrent Neural Networks

Nanda H Krishna

Colin Bredenberg

Daniel Levenstein

Blake Aaron Richards

Guillaume Lajoie

During periods of quiescence, such as sleep, neural activity in many brain circuits resembles that observed during periods of task engagemen… (voir plus)t. However, the precise conditions under which task-optimized networks can autonomously reactivate the same network states responsible for online behavior is poorly understood. In this study, we develop a mathematical framework that outlines sufficient conditions for the emergence of neural reactivation in circuits that encode features of smoothly varying stimuli. We demonstrate mathematically that noisy recurrent networks optimized to track environmental state variables using change-based sensory information naturally develop denoising dynamics, which, in the absence of input, cause the network to revisit state configurations observed during periods of online activity. We validate our findings using numerical experiments on two canonical neuroscience tasks: spatial position estimation based on self-motion cues, and head direction estimation based on angular velocity cues. Overall, our work provides theoretical support for modeling offline reactivation as an emergent consequence of task optimization in noisy neural circuits.

2024-01-15

International Conference on Learning Representations (poster)

Gintare Karolina Dziugaite

The Cost of Scaling Down Large Language Models: Reducing Model Size Affects Memory before In-context Learning.

Tian Jin

Nolan Clement

Xin Dong

Vaishnavh Nagarajan

Michael Carbin

Jonathan Ragan-kelley

2024-01-15

ICLR.cc/2024/Conference (poster)

Towards Foundation Models for Knowledge Graph Reasoning

Hesham Mostafa

Foundation models in language and vision have the ability to run inference on any textual and visual inputs thanks to the transferable repre… (voir plus)sentations such as a vocabulary of tokens in language. Knowledge graphs (KGs) have different entity and relation vocabularies that generally do not overlap. The key challenge of designing foundation models on KGs is to learn such transferable representations that enable inference on any graph with arbitrary entity and relation vocabularies. In this work, we make a step towards such foundation models and present ULTRA, an approach for learning universal and transferable graph representations. ULTRA builds relational representations as a function conditioned on their interactions. Such a conditioning strategy allows a pre-trained ULTRA model to inductively generalize to any unseen KG with any relation vocabulary and to be fine-tuned on any graph. Conducting link prediction experiments on 57 different KGs, we find that the zero-shot inductive inference performance of a single pre-trained ULTRA model on unseen graphs of various sizes is often on par or better than strong baselines trained on specific graphs. Fine-tuning further boosts the performance.

2024-01-15

ICLR.cc/2024/Conference (poster)

Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models

Pablo Pernias

Dominic Rampas

Mats Leon Richter

Christopher Pal

Marc Aubreville

2024-01-15

ICLR.cc/2024/Conference (présentation orale)

BCG immunization induces CX3CR1hi effector memory T cells to provide cross-protection via IFN-γ-mediated trained immunity.

Kim A. Tran

Erwan Pernet

Mina Sadeghi

Jeffrey Downey

Julia Chronopoulos

Elizabeth Lapshina

Oscar Tsai

Eva Kaufmann

Jun Ding

Maziar Divangahi

2024-01-14

Nature Immunology (publié)

Self-Supervised Anomaly Detection: A Survey and Outlook

Hadi Hojjati

Thi Kieu Khanh Ho

Naregs Armanfard

Anomaly detection (AD) plays a crucial role in various domains, including cybersecurity, finance, and healthcare, by identifying patterns or… (voir plus) events that deviate from normal behaviour. In recent years, significant progress has been made in this field due to the remarkable growth of deep learning models. Notably, the advent of self-supervised learning has sparked the development of novel AD algorithms that outperform the existing state-of-the-art approaches by a considerable margin. This paper aims to provide a comprehensive review of the current methodologies in self-supervised anomaly detection. We present technical details of the standard methods and discuss their strengths and drawbacks. We also compare the performance of these models against each other and other state-of-the-art anomaly detection models. Finally, the paper concludes with a discussion of future directions for self-supervised anomaly detection, including the development of more effective and efficient algorithms and the integration of these techniques with other related fields, such as multi-modal learning.

2024-01-14

Neural Networks (inconnu)

Computational pathology: A survey review and the way forward

Mahdi S. Hosseini

Babak Ehteshami Bejnordi

Vincent Quoc-Huy Trinh

Danial Hasan

Xingwen Li

Taehyo Kim

Haochen Zhang

Theodore Wu

Kajanan Chinniah

Sina Maghsoudlou

Ryan Zhang

Stephen Yang

Jiadai Zhu

Lyndon Chan

Samir Khaki

Andrei Buin

Fatemeh Chaji

Ala Salehi

Alejandra Zambrano Luna

Bich Ngoc Nguyen … (voir 2 de plus)

Dimitris Samaras

Konstantinos N. Plataniotis

2024-01-13

Journal of Pathology Informatics (publié)

Assessing the quality and value of metabolic chart data for capturing core outcomes for pediatric medium-chain acyl-CoA dehydrogenase (MCAD) deficiency

Ryan Iverson

Monica Taljaard

Michael T. Geraghty

Michael Pugliese

Kylie Tingley

Doug Coyle

Jonathan B. Kronick

Kumanan Wilson

Valerie Austin

Catherine Brunel-Guitton

Daniela Buhas

Nancy J. Butcher

Alicia K. J. Chan

Sarah Dyack

Sharan Goobie

Cheryl Greenberg

Shailly Jain-Ghai

Michal Inbar-Feigenberg

Natalya Karp

Mariya Kozenko … (voir 30 de plus)

Erica Langley

Matthew Lines

Julian Little

Jennifer MacKenzie

Bruno Maranda

Saadet Mercimek-Andrews

Aizeddin Mhanni

John J. Mitchell

Laura Nagy

Martin Offringa

Amy Pender

Murray Potter

Chitra Prasad

Suzanne Ratko

Ramona Salvarinova

Andreas Schulze

Komudi Siriwardena

Neal Sondheimer

Rebecca Sparkes

Sylvia Stockler-Ipsiroglu

Kendra Tapscott

Yannis Trakadis

Lesley Turner

Clara Van Karnebeek

Anthony Vandersteen

Jagdeep S. Walia

Brenda J. Wilson

Andrea C. Yu

Beth K. Potter

Pranesh Chakraborty

2024-01-12

BMC Pediatrics (publié)

Combining Confidence Elicitation and Sample-based Methods for Uncertainty Quantification in Misinformation Mitigation

Mauricio Rivera

Jean-François Godbout

Reihaneh Rabbany

Kellin Pelrine

2024-01-12

ArXiv (prépublication)

Community-based Reconstruction and Simulation of a Full-scale Model of Region CA1 of Rat Hippocampus

Armando Romani

Alberto Antonietti

Davide Bella

Julian Budd

Elisabetta Giacalone

Kerem Kurban

Sára Sáray

Marwan Abdellah

Alexis Arnaudon

Elvis Boci

Cristina Colangelo

Jean-Denis Courcol

Thomas Delemontex

András Ecker

Joanne Falck

Cyrille Favreau

Michael Gevaert

Juan B. Hernando

Joni Herttuainen

Genrich Ivaska … (voir 28 de plus)

Lida Kanari

Anna-Kristin Kaufmann

James King

Pramod Kumbhar

Sigrun Lange

Huanxiang Lu

Carmen Alina Lupascu

Rosanna Migliore

Fabien Petitjean

Judit Planas

Pranav Rai

Srikanth Ramaswamy

Michael W. Reimann

Juan Luis Riquelme

Nadir Román Guerrero

Ying Shi

Vishal Sood

Mohameth François Sy

Werner Van Geit

Liesbeth Vanherpe

Tamás F. Freund

Audrey Mercer

Eilif B. Muller

Felix Schürmann

Alex M. Thomson

Michele Migliore

Szabolcs Káli

Henry Markram

The CA1 region of the hippocampus is one of the most studied regions of the rodent brain, thought to play an important role in cognitive fun… (voir plus)ctions such as memory and spatial navigation. Despite a wealth of experimental data on its structure and function, it has been challenging to reconcile information obtained from diverse experimental approaches. To address this challenge, we present a community-driven, full-scale in silico model of the rat CA1 that integrates a broad range of experimental data, from synapse to network, including the reconstruction of its principal afferents, the Schaffer collaterals, and a model of the effects that acetylcholine has on the system. We tested and validated each model component and the final network model, and made input data, assumptions, and strategies explicit and transparent. The unique flexibility of the model allows scientists to address a range of scientific questions. In this article, we describe the methods used to set up simulations that reproduce and extend in vitro and in vivo experiments. Among several applications in the article, we focus on theta rhythm, a prominent hippocampal oscillation associated with various behavioral correlates and use our computer model to reproduce and reconcile experimental findings. Finally, we make data, code and model available through the hippocampushub.eu portal, which also provides an extensive set of analyses of the model and a user-friendly interface to facilitate adoption and usage. This neuroscience community-driven model represents a valuable tool for integrating diverse experimental data and provides a foundation for further research into the complex workings of the hippocampal CA1 region.

2024-01-12

bioRxiv (prépublication)

Comparing GPT-4 and Open-Source Language Models in Misinformation Mitigation

Tyler Vergho

Jean-François Godbout

Reihaneh Rabbany

Kellin Pelrine

Recent large language models (LLMs) have been shown to be effective for misinformation detection. However, the choice of LLMs for experiment… (voir plus)s varies widely, leading to uncertain conclusions. In particular, GPT-4 is known to be strong in this domain, but it is closed source, potentially expensive, and can show instability between different versions. Meanwhile, alternative LLMs have given mixed results. In this work, we show that Zephyr-7b presents a consistently viable alternative, overcoming key limitations of commonly used approaches like Llama-2 and GPT-3.5. This provides the research community with a solid open-source option and shows open-source models are gradually catching up on this task. We then highlight how GPT-3.5 exhibits unstable performance, such that this very widely used model could provide misleading results in misinformation detection. Finally, we validate new tools including approaches to structured output and the latest version of GPT-4 (Turbo), showing they do not compromise performance, thus unlocking them for future research and potentially enabling more complex pipelines for misinformation mitigation.

2024-01-11

ArXiv (prépublication)