Publications

Towards a “universal translator” for neural dynamics at single-cell, single-spike resolution

Yizi Zhang

Yanchen Wang

Donato M. Jiménez-Benetó

Zixuan Wang

Mehdi Azabou

Blake Richards

Renee Tung

Olivier Winter

The International Brain Laboratory

Eva Dyer

Liam Paninski

Cole Hurwitz

Neuroscience research has made immense progress over the last decade, but our understanding of the brain remains fragmented and piecemeal: t… (see more)he dream of probing an arbitrary brain region and automatically reading out the information encoded in its neural activity remains out of reach. In this work, we build towards a first foundation model for neural spiking data that can solve a diverse set of tasks across multiple brain areas. We introduce a novel self-supervised modeling approach for population activity in which the model alternates between masking out and reconstructing neural activity across different time steps, neurons, and brain regions. To evaluate our approach, we design unsupervised and supervised prediction tasks using the International Brain Laboratory repeated site dataset, which is comprised of Neuropixels recordings targeting the same brain locations across 48 animals and experimental sessions. The prediction tasks include single-neuron and region-level activity prediction, forward prediction, and behavior decoding. We demonstrate that our multi-task-masking (MtM) approach significantly improves the performance of current state-of-the-art population models and enables multi-task learning. We also show that by training on multiple animals, we can improve the generalization ability of the model to unseen animals, paving the way for a foundation model of the brain at single-cell, single-spike resolution.

2024-09-24

NeurIPS.cc/2024/Conference (poster)

doi.org

openreview.net

VisMin: Visual Minimal-Change Understanding

Fine-grained understanding of objects, attributes, and relationships between objects is crucial for visual-language models (VLMs). Existing … (see more)benchmarks primarily focus on evaluating VLMs' capability to distinguish between two very similar captions given an image. In this paper, we introduce a new, challenging benchmark termed Visual Minimal-Change Understanding (VisMin), which requires models to predict the correct image-caption match given two images and two captions. The image pair and caption pair contain minimal changes, i.e., only one aspect changes at a time from among the following: object, attribute, count, and spatial relation. These changes test the models' understanding of objects, attributes (such as color, material, shape), counts, and spatial relationships between objects. We built an automatic framework using large language models and diffusion models, followed by a rigorous 4-step verification process by human annotators. Empirical experiments reveal that current VLMs exhibit notable deficiencies in understanding spatial relationships and counting abilities. We also generate a large-scale training dataset to finetune CLIP and Idefics2, showing significant improvements in fine-grained understanding across benchmarks and in CLIP's general image-text alignment. We release all resources, including the benchmark, training data, and finetuned model checkpoints, at https://vismin.net/.

2024-09-24

NeurIPS.cc/2024/Conference (poster)

doi.org

openreview.net

Wasserstein Distributionally Robust Optimization Through the Lens of Structural Causal Models and Individual Fairness

Ahmad-reza Ehyaei

Golnoosh Farnadi

Samira Samadi

In recent years, Wasserstein Distributionally Robust Optimization (DRO) has garnered substantial interest for its efficacy in data-driven de… (see more)cision-making under distributional uncertainty. However, limited research has explored the application of DRO to address individual fairness concerns, particularly when considering causal structures and sensitive attributes in learning problems. To address this gap, we first formulate the DRO problem from causality and individual fairness perspectives. We then present the DRO dual formulation as an efficient tool to convert the DRO problem into a more tractable and computationally efficient form. Next, we characterize the closed form of the approximate worst-case loss quantity as a regularizer, eliminating the max-step in the min-max DRO problem. We further estimate the regularizer in more general cases and explore the relationship between DRO and classical robust optimization. Finally, by removing the assumption of a known structural causal model, we provide finite sample error bounds when designing DRO with empirical distributions and estimated causal structures to ensure efficiency and robust learning.

2024-09-24

NeurIPS.cc/2024/Conference (poster)

doi.org

openreview.net

When is an Embedder More Promising than Another?

Maxime Darrin

Philippe Formont

Ismail Ben Ayed

Jackie Chi Kit Cheung

Pablo Piantanida

Embedders play a central role in machine learning, projecting any object into numerical representations that can, in turn, be leveraged to p… (see more)erform various downstream tasks. The evaluation of embedding models typically depends on domain-specific empirical approaches utilizing downstream tasks, primarily because of the lack of a standardized framework for comparison. However, acquiring adequately large and representative datasets for conducting these assessments is not always viable and can prove to be prohibitively expensive and time-consuming. In this paper, we present a unified approach to evaluate embedders. First, we establish theoretical foundations for comparing embedding models, drawing upon the concepts of sufficiency and informativeness. We then leverage these concepts to devise a tractable comparison criterion (information sufficiency), leading to a task-agnostic and self-supervised ranking procedure. We demonstrate experimentally that our approach aligns closely with the capability of embedding models to facilitate various downstream tasks in both natural language processing and molecular biology. This effectively offers practitioners a valuable tool for prioritizing model trials.

2024-09-24

NeurIPS.cc/2024/Conference (poster)

doi.org

openreview.net

Frequency-based View Selection in Gaussian Splatting Reconstruction

Monica Li

Pierre-Yves Lajoie

Giovanni Beltrame

Three-dimensional reconstruction is a fundamental problem in robotics perception. We examine the problem of active view selection to perform… (see more) 3D Gaussian Splatting reconstructions with as few input images as possible. Although 3D Gaussian Splatting has made significant progress in image rendering and 3D reconstruction, the quality of the reconstruction is strongly impacted by the selection of 2D images and the estimation of camera poses through Structure-from-Motion (SfM) algorithms. Current methods to select views that rely on uncertainties from occlusions, depth ambiguities, or neural network predictions directly are insufficient to handle the issue and struggle to generalize to new scenes. By ranking the potential views in the frequency domain, we are able to effectively estimate the potential information gain of new viewpoints without ground truth data. By overcoming current constraints on model architecture and efficacy, our method achieves state-of-the-art results in view selection, demonstrating its potential for efficient image-based 3D reconstruction.

2024-09-23

ArXiv (preprint)

doi.org

arxiv.org

Derivation and validation of indices incorporating vasopressor dose and blood pressure values over time

Alain Gervais

François Lamontagne

Jean-Baptiste Michaud

KJ Neill

Adhikari

Jean-Michel Pagé

Marie-Hélène Masse

Michael O. Harhay

Michael Chassé

Félix Lamontagne

Katia Laforge

Alexandra Fortin

Marc-André Leclair

Simon Lévesque

Marie-Pier Domingue

Neda Momenzadeh

Martin Vallières

Ruxandra Pinto

Maxime Morin-Lavoie

Francis Carter … (see 2 more)

Félix Camirand Lemyre

MD MSc. François Lamontagne

Rationale The blood pressure value below which the benefits of vasopressors clearly outweigh their disadvantages is uncertain. Objectives Th… (see more)e main objective of this analysis was to investigate the statistical properties and potential utility of indices estimating the vasopressor dose-rates as a function of blood pressure values over time. Methods In this single-center observational study, we collected blood pressure values from intensive care unit (ICU) monitors and norepinephrine dose-rates from infusion pumps corresponding to a derivation and a validation cohort. Patients included in each cohort were 18 years or older and received norepinephrine in the ICU. We defined and derived indices corresponding to vasopressor therapy above (>65 mmHg) and below (60 mmHg) targets. We report the distribution of both indices over time from both cohorts as well as their associations with hospital mortal

2024-09-22

medRxiv (preprint)

doi.org

Not Only the Last-Layer Features for Spurious Correlations: All Layer Deep Feature Reweighting

Humza Wajid Hameed

Geraldin Nanfack

Eugene Belilovsky

Spurious correlations are a major source of errors for machine learning models, in particular when aiming for group-level fairness. It has b… (see more)een recently shown that a powerful approach to combat spurious correlations is to re-train the last layer on a balanced validation dataset, isolating robust features for the predictor. However, key attributes can sometimes be discarded by neural networks towards the last layer. In this work, we thus consider retraining a classifier on a set of features derived from all layers. We utilize a recently proposed feature selection strategy to select unbiased features from all the layers. We observe this approach gives significant improvements in worst-group accuracy on several standard benchmarks.

2024-09-22

ArXiv (preprint)

doi.org

arxiv.org

Protein Language Models: Is Scaling Necessary?

Quentin Fournier

Robert M. Vernon

Almer Van Der Sloot

Benjamin Schulz

Sarath Chandar

Christopher James Langmead

Public protein sequence databases contain samples from the fitness landscape explored by nature. Protein language models (pLMs) pre-trained … (see more)on these sequences aim to capture this landscape for tasks like property prediction and protein design. Following the same trend as in natural language processing, pLMs have continuously been scaled up. However, the premise that scale leads to better performance assumes that source databases provide an accurate representation of the underlying fitness landscape, which is likely false. By developing an efficient codebase, designing a modern architecture, and addressing data quality concerns such as sample bias, we introduce AMPLIFY, a best-in-class pLM that is orders of magnitude less expensive to train and deploy than previous models. Furthermore, to support the scientific community and democratize the training of pLMs, we have open-sourced AMPLIFY’s pre-training codebase, data, and model checkpoints.

2024-09-22

bioRxiv (preprint)

doi.org

Self Supervised Dictionary Learning Using Kernel Matching

Shubham Choudhary

Paul Masset

Demba Ba

We introduce a self supervised framework for learning representations in the context of dictionary learning. We cast the problem as a kernel… (see more) matching task between the input and the representation space, with constraints on the latent kernel. By adjusting these constraints, we demonstrate how the framework can adapt to different learning objectives. We then formulate a novel Alternate Direction Method of Multipli-ers (ADMM) based algorithm to solve the optimization problem and connect the dynamics to classical alternate minimization techniques. This approach offers a unique way of learning representations with kernel constraints, that enable us implicitly learn a generative map for the data from the learned representations which can have broad applications in representation learning tasks both in machine learning and neuro-science.

2024-09-21

International Workshop on Machine Learning for Signal Processing (published)

doi.org

Al content detection in the emerging information ecosystem: new obligations for media and tech companies

Alistair Knott

Dino Pedreschi

Toshiya Jitsuzumi

Susan Leavy

David Eyers

Tapabrata Chakraborti

Andrew Trotman

Sundar Sundareswaran

Ricardo Baeza-Yates

Przemyslaw Biecek

Adrian Weller

Paul D. Teal

Subhadip Basu

Mehmet Haklidir

Virginia Morini

Stuart Russell

Yoshua Bengio

2024-09-20

Ethics and Information Technology (published)

doi.org

ToxiSight: Insights Towards Detected Chat Toxicity

Zachary Yang

Domenico Tullo

Reihaneh Rabbany

We present a comprehensive explainability dashboard designed for in-game chat toxicity. This dashboard integrates various existing explainab… (see more)le AI (XAI) techniques, including token importance analysis, model output visualization, and attribution to the training dataset. It also provides insights through the closest positive and negative examples, facilitating a deeper understanding and potential correction of the training data. Additionally, the dashboard includes word sense analysis—particularly useful for new moderators—and offers free-text explanations for both positive and negative predictions. This multi-faceted approach enhances the interpretability and transparency of toxicity detection models.

2024-09-20

EMNLP/2024/Workshop/BlackBoxNLP (accepted)

openreview.net