Publications

Str2str: A Score-Based Framework for Zero-Shot Protein Conformation Sampling

Bozitao Zhong

The dynamic nature of proteins is crucial for determining their biological functions and properties, for which Monte Carlo (MC) and molecula… (voir plus)r dynamics (MD) simulations stand as predominant tools to study such phenomena. By utilizing empirically derived force fields, MC or MD simulations explore the conformational space through numerically evolving the system via Markov chain or Newtonian mechanics. However, the high-energy barrier of the force fields can hamper the exploration of both methods by the rare event, resulting in inadequately sampled ensemble without exhaustive running. Existing learning-based approaches perform direct sampling yet heavily rely on target-specific simulation data for training, which suffers from high data acquisition cost and poor generalizability. Inspired by simulated annealing, we propose Str2Str, a novel structure-to-structure translation framework capable of zero-shot conformation sampling with roto-translation equivariant property. Our method leverages an amortized denoising score matching objective trained on general crystal structures and has no reliance on simulation data during both training and inference. Experimental results across several benchmarking protein systems demonstrate that Str2Str outperforms previous state-of-the-art generative structure prediction models and can be orders of magnitude faster compared to long MD simulations. Our open-source implementation is available at https://github.com/lujiarui/Str2Str

2024-01-15

ICLR.cc/2024/Conference (poster)

doi.org

openreview.net

Sufficient Conditions for Offline Reactivation in Recurrent Neural Networks

Nanda H Krishna

Colin Bredenberg

Daniel Levenstein

Blake Aaron Richards

Guillaume Lajoie

During periods of quiescence, such as sleep, neural activity in many brain circuits resembles that observed during periods of task engagemen… (voir plus)t. However, the precise conditions under which task-optimized networks can autonomously reactivate the same network states responsible for online behavior is poorly understood. In this study, we develop a mathematical framework that outlines sufficient conditions for the emergence of neural reactivation in circuits that encode features of smoothly varying stimuli. We demonstrate mathematically that noisy recurrent networks optimized to track environmental state variables using change-based sensory information naturally develop denoising dynamics, which, in the absence of input, cause the network to revisit state configurations observed during periods of online activity. We validate our findings using numerical experiments on two canonical neuroscience tasks: spatial position estimation based on self-motion cues, and head direction estimation based on angular velocity cues. Overall, our work provides theoretical support for modeling offline reactivation as an emergent consequence of task optimization in noisy neural circuits.

2024-01-15

International Conference on Learning Representations (poster)

doi.org

openreview.net

Synaptic Weight Distributions Depend on the Geometry of Plasticity

Blake Aaron Richards

A growing literature in computational neuroscience leverages gradient descent and learning algorithms that approximate it to study synaptic … (voir plus)plasticity in the brain. However, the vast majority of this work ignores a critical underlying assumption: the choice of distance for synaptic changes - i.e. the geometry of synaptic plasticity. Gradient descent assumes that the distance is Euclidean, but many other distances are possible, and there is no reason that biology necessarily uses Euclidean geometry. Here, using the theoretical tools provided by mirror descent, we show that the distribution of synaptic weights will depend on the geometry of synaptic plasticity. We use these results to show that experimentally-observed log-normal weight distributions found in several brain areas are not consistent with standard gradient descent (i.e. a Euclidean geometry), but rather with non-Euclidean distances. Finally, we show that it should be possible to experimentally test for different synaptic geometries by comparing synaptic weight distributions before and after learning. Overall, our work shows that the current paradigm in theoretical work on synaptic plasticity that assumes Euclidean synaptic geometry may be misguided and that it should be possible to experimentally determine the true geometry of synaptic plasticity in the brain.

2024-01-15

ICLR.cc/2024/Conference (spotlight)

doi.org

openreview.net

TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series

Arjun Ashok

Étienne Marcotte

Valentina Zantedeschi

Nicolas Chapados

Alexandre Drouin

We introduce a new model for multivariate probabilistic time series prediction, designed to flexibly address a range of tasks including fore… (voir plus)casting, interpolation, and their combinations. Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS), wherein the number of distributional parameters now scales linearly with the number of variables instead of factorially. The new objective requires the introduction of a training curriculum, which goes hand-in-hand with necessary changes to the original architecture. We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks, while maintaining the flexibility of prior work, such as seamless handling of unaligned and unevenly-sampled time series. Code is made available at https://github.com/ServiceNow/TACTiS.

2024-01-15

ICLR.cc/2024/Conference (poster)

doi.org

openreview.net

The Curse of Diversity in Ensemble-Based Exploration

We uncover a surprising phenomenon in deep reinforcement learning: training a diverse ensemble of data-sharing agents -- a well-established … (voir plus)exploration strategy -- can significantly impair the performance of the individual ensemble members when compared to standard single-agent training. Through careful analysis, we attribute the degradation in performance to the low proportion of self-generated data in the shared training data for each ensemble member, as well as the inefficiency of the individual ensemble members to learn from such highly off-policy data. We thus name this phenomenon the curse of diversity. We find that several intuitive solutions -- such as a larger replay buffer or a smaller ensemble size -- either fail to consistently mitigate the performance loss or undermine the advantages of ensembling. Finally, we demonstrate the potential of representation learning to counteract the curse of diversity with a novel method named Cross-Ensemble Representation Learning (CERL) in both discrete and continuous control domains. Our work offers valuable insights into an unexpected pitfall in ensemble-based exploration and raises important caveats for future applications of similar approaches.

2024-01-15

ICLR.cc/2024/Conference (poster)

doi.org

openreview.net

On the Stability of Iterative Retraining of Generative Models on Their Own Data

Avishek (Joey) Bose

Deep generative models have made tremendous progress in modeling complex data, often exhibiting generation quality that surpasses a typical … (voir plus)human's ability to discern the authenticity of samples. Undeniably, a key driver of this success is enabled by the massive amounts of web-scale data consumed by these models. Due to these models' striking performance and ease of availability, the web will inevitably be increasingly populated with synthetic content. Such a fact directly implies that future iterations of generative models will be trained on both clean and artificially generated data from past models. In this paper, we develop a framework to rigorously study the impact of training generative models on mixed datasets -- from classical training on real data to self-consuming generative models trained on purely synthetic data. We first prove the stability of iterative training under the condition that the initial generative models approximate the data distribution well enough and the proportion of clean training data (w.r.t. synthetic data) is large enough. We empirically validate our theory on both synthetic and natural images by iteratively training normalizing flows and state-of-the-art diffusion models on CIFAR10 and FFHQ.

2024-01-15

ICLR.cc/2024/Conference (spotlight)

doi.org

openreview.net

Towards Foundation Models for Knowledge Graph Reasoning

Hesham Mostafa

Foundation models in language and vision have the ability to run inference on any textual and visual inputs thanks to the transferable repre… (voir plus)sentations such as a vocabulary of tokens in language. Knowledge graphs (KGs) have different entity and relation vocabularies that generally do not overlap. The key challenge of designing foundation models on KGs is to learn such transferable representations that enable inference on any graph with arbitrary entity and relation vocabularies. In this work, we make a step towards such foundation models and present ULTRA, an approach for learning universal and transferable graph representations. ULTRA builds relational representations as a function conditioned on their interactions. Such a conditioning strategy allows a pre-trained ULTRA model to inductively generalize to any unseen KG with any relation vocabulary and to be fine-tuned on any graph. Conducting link prediction experiments on 57 different KGs, we find that the zero-shot inductive inference performance of a single pre-trained ULTRA model on unseen graphs of various sizes is often on par or better than strong baselines trained on specific graphs. Fine-tuning further boosts the performance.

2024-01-15

ICLR.cc/2024/Conference (poster)

doi.org

openreview.net

BCG immunization induces CX3CR1hi effector memory T cells to provide cross-protection via IFN-γ-mediated trained immunity.

Kim A. Tran

Erwan Pernet

Mina Sadeghi

Jeffrey Downey

Julia Chronopoulos

Elizabeth Lapshina

Oscar Tsai

Eva Kaufmann

Jun Ding

Maziar Divangahi

2024-01-14

Nature Immunology (publié)

doi.org

Self-Supervised Anomaly Detection: A Survey and Outlook

Hadi Hojjati

Thi Kieu Khanh Ho

Naregs Armanfard

Anomaly detection (AD) plays a crucial role in various domains, including cybersecurity, finance, and healthcare, by identifying patterns or… (voir plus) events that deviate from normal behaviour. In recent years, significant progress has been made in this field due to the remarkable growth of deep learning models. Notably, the advent of self-supervised learning has sparked the development of novel AD algorithms that outperform the existing state-of-the-art approaches by a considerable margin. This paper aims to provide a comprehensive review of the current methodologies in self-supervised anomaly detection. We present technical details of the standard methods and discuss their strengths and drawbacks. We also compare the performance of these models against each other and other state-of-the-art anomaly detection models. Finally, the paper concludes with a discussion of future directions for self-supervised anomaly detection, including the development of more effective and efficient algorithms and the integration of these techniques with other related fields, such as multi-modal learning.

2024-01-14

Neural Networks (inconnu)

doi.org

arxiv.org

Computational pathology: A survey review and the way forward

Mahdi S. Hosseini

Babak Ehteshami Bejnordi

Vincent Quoc-Huy Trinh

Danial Hasan

Xingwen Li

Taehyo Kim

Haochen Zhang

Theodore Wu

Kajanan Chinniah

Sina Maghsoudlou

Ryan Zhang

Stephen Yang

Jiadai Zhu

Lyndon Chan

Samir Khaki

Andrei Buin

Fatemeh Chaji

Ala Salehi

Alejandra Zambrano Luna

Bich Ngoc Nguyen … (voir 2 de plus)

Dimitris Samaras

Konstantinos N. Plataniotis

2024-01-13

Journal of Pathology Informatics (publié)

doi.org

arxiv.org

Assessing the quality and value of metabolic chart data for capturing core outcomes for pediatric medium-chain acyl-CoA dehydrogenase (MCAD) deficiency

Ryan Iverson

Monica Taljaard

Michael T. Geraghty

Michael Pugliese

Kylie Tingley

Doug Coyle

Jonathan B. Kronick

Kumanan Wilson

Valerie Austin

Catherine Brunel-Guitton

Daniela Buhas

Nancy J. Butcher

Alicia K. J. Chan

Sarah Dyack

Sharan Goobie

Cheryl Greenberg

Shailly Jain-Ghai

Michal Inbar-Feigenberg

Natalya Karp

Mariya Kozenko … (voir 30 de plus)

Erica Langley

Matthew Lines

Julian Little

Jennifer MacKenzie

Bruno Maranda

Saadet Mercimek-Andrews

Aizeddin Mhanni

John J. Mitchell

Laura Nagy

Martin Offringa

Amy Pender

Murray Potter

Chitra Prasad

Suzanne Ratko

Ramona Salvarinova

Andreas Schulze

Komudi Siriwardena

Neal Sondheimer

Rebecca Sparkes

Sylvia Stockler-Ipsiroglu

Kendra Tapscott

Yannis Trakadis

Lesley Turner

Clara Van Karnebeek

Anthony Vandersteen

Jagdeep S. Walia

Brenda J. Wilson

Andrea C. Yu

Beth K. Potter

Pranesh Chakraborty

2024-01-12

BMC Pediatrics (publié)

doi.org

Combining Confidence Elicitation and Sample-based Methods for Uncertainty Quantification in Misinformation Mitigation

Mauricio Rivera

Jean-François Godbout

Reihaneh Rabbany

Kellin Pelrine

2024-01-12

ArXiv (prépublication)

doi.org

arxiv.org

Mila sur Udemy

Publications du Fellowship en politiques de l'IA

La plateforme Mila Ventures

Publications

Mila sur Udemy

Publications du Fellowship en politiques de l'IA

La plateforme Mila Ventures

Mots-clés populaires:

Publications