Smita Krishnaswamy

Github

Google Scholar

Publications

Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy

Danqi Liao

Chen Liu

Benjamin W Christensen

Maximilian Nickel

Ian Adelstein

Entropy and mutual information in neural networks provide rich information on the learning process, but they have proven difficult to comput… (see more)e reliably in high dimensions. Indeed, in noisy and high-dimensional data, traditional estimates in ambient dimensions approach a fixed entropy and are prohibitively hard to compute. To address these issues, we leverage data geometry to access the underlying manifold and reliably compute these information-theoretic measures. Specifically, we define diffusion spectral entropy (DSE) in neural representations of a dataset as well as diffusion spectral mutual information (DSMI) between different variables representing data. First, we show that they form noise-resistant measures of intrinsic dimensionality and relationship strength in high-dimensional simulated data that outperform classic Shannon entropy, nonparametric estimation, and mutual information neural estimation (MINE). We then study the evolution of representations in classification networks with supervised learning, self-supervision, or overfitting. We observe that (1) DSE of neural representations increases during training; (2) DSMI with the class label increases during generalizable learning but stays stagnant during overfitting; (3) DSMI with the input signal shows differing trends: on MNIST it increases, while on CIFAR-10 and STL-10 it decreases. Finally, we show that DSE can be used to guide better network initialization and that DSMI can be used to predict downstream classification accuracy across 962 models on ImageNet.

2023-12-31

CISS (published)

Learnable Filters for Geometric Scattering Modules

Frederik Wenkel

Dhananjay Bhaskar

Kincaid MacDonald

Jackson Grady

Michael Perlmutter

2023-12-31

IEEE Transactions on Signal Processing (published)

arxiv.org

Low-Dimensional Embeddings of High-Dimensional Data: Algorithms and Applications (Dagstuhl Seminar 24122).

Dmitry Kobak

Fred Hamprecht

Gal Mishne

Sebastian Damrich

2023-12-31

Dagstuhl Reports (published)

Neural FIM for learning Fisher information metrics from point cloud data

Oluwadamilola Fasina

Yanlei Zhang

Maximilian Nickel

Ian Adelstein

Although data diffusion embeddings are ubiquitous in unsupervised learning and have proven to be a viable technique for uncovering the under… (see more)lying intrinsic geometry of data, diffusion embeddings are inherently limited due to their discrete nature. To this end, we propose neural FIM, a method for computing the Fisher information metric (FIM) from point cloud data - allowing for a continuous manifold model for the data. Neural FIM creates an extensible metric space from discrete point cloud data such that information from the metric can inform us of manifold characteristics such as volume and geodesics. We demonstrate Neural FIM's utility in selecting parameters for the PHATE visualization method as well as its ability to obtain information pertaining to local volume illuminating branching points and cluster centers embeddings of a toy dataset and two single-cell datasets of IPSC reprogramming and PBMCs (immune cells).

2023-07-02

Proceedings of the 40th International Conference on Machine Learning (published)

proceedings.mlr.press

HSV-2 triggers upregulation of MALAT1 in CD4+ T cells and promotes HIV latency reversal

Carl A. Pierce

Lip Nam Loh

Holly Steach

Natalia Cheshenko

Paula Preston-Hurlburt

Fengrui Zhang

Stephanie Stransky

Leah Kravets

Simone Sidoli

William Philbrick

Michel Nassar

Kevan C. Herold

Betsy C. Herold

Herpes simplex virus type 2 (HSV-2) coinfection is associated with increased HIV-1 viral loads and expanded tissue reservoirs, but the mecha… (see more)nisms are not well defined. HSV-2 recurrences result in an influx of activated CD4+ T cells to sites of viral replication and an increase in activated CD4+ T cells in peripheral blood. We hypothesized that HSV-2 induces changes in these cells that facilitate HIV-1 reactivation and replication and tested this hypothesis in human CD4+ T cells and 2D10 cells, a model of HIV-1 latency. HSV-2 promoted latency reversal in HSV-2–infected and bystander 2D10 cells. Bulk and single-cell RNA-Seq studies of activated primary human CD4+ T cells identified decreased expression of HIV-1 restriction factors and increased expression of transcripts including MALAT1 that could drive HIV replication in both the HSV-2–infected and bystander cells. Transfection of 2D10 cells with VP16, an HSV-2 protein that regulates transcription, significantly upregulated MALAT1 expression, decreased trimethylation of lysine 27 on histone H3 protein, and triggered HIV latency reversal. Knockout of MALAT1 from 2D10 cells abrogated the response to VP16 and reduced the response to HSV-2 infection. These results demonstrate that HSV-2 contributes to HIV-1 reactivation through diverse mechanisms, including upregulation of MALAT1 to release epigenetic silencing.

2023-05-31

The Journal of Clinical Investigation (published)

Graph Fourier MMD for Signals on Graphs

Samuel Leone

Aarthi Venkat

While numerous methods have been proposed for computing distances between probability distributions in Euclidean space, relatively little at… (see more)tention has been given to computing such distances for distributions on graphs. However, there has been a marked increase in data that either lies on graph (such as protein interaction networks) or can be modeled as a graph (single cell data), particularly in the biomedical sciences. Thus, it becomes important to find ways to compare signals defined on such graphs. Here, we propose Graph Fourier MMD (GFMMD), a novel distance between distributions and signals on graphs. GFMMD is defined via an optimal witness function that is both smooth on the graph and maximizes difference in expectation between the pair of distributions on the graph. We find an analytical solution to this optimization problem as well as an embedding of distributions that results from this method. We also prove several properties of this method including scale invariance and applicability to disconnected graphs. We showcase it on graph benchmark datasets as well on single cell RNA-sequencing data analysis. In the latter, we use the GFMMD-based gene embeddings to find meaningful gene clusters. We also propose a novel type of score for gene selection called "gene localization score" which helps select genes for cellular state space characterization.

2023-05-20

SampTA/2023/Conference (published)

Single-cell analysis reveals inflammatory interactions driving macular degeneration

Manik Kuchroo

Marcello DiStasio

Eric Song

Eda Calapkulu

Le Zhang

Maryam Ige

Amar H. Sheth

Abdelilah Majdoubi

Madhvi Menon

Abhinav Godavarthi

Yu Xing

Scott Gigante

Holly Steach

Jessie Huang

Janhavi Narain

Kisung You

George Mourgkos … (see 6 more)

Rahul M. Dhodapkar

Matthew J. Hirn

Bastian Rieck

Brian P. Hafler

Due to commonalities in pathophysiology, age-related macular degeneration (AMD) represents a uniquely accessible model to investigate thera… (see more)pies for neurodegenerative diseases, leading us to examine whether pathways of disease progression are shared across neurodegenerative conditions. Here we use single-nucleus RNA sequencing to profile lesions from 11 postmortem human retinas with age-related macular degeneration and 6 control retinas with no history of retinal disease. We create a machine-learning pipeline based on recent advances in data geometry and topology and identify activated glial populations enriched in the early phase of disease. Examining single-cell data from Alzheimer’s disease and progressive multiple sclerosis with our pipeline, we find a similar glial activation profile enriched in the early phase of these neurodegenerative diseases. In late-stage age-related macular degeneration, we identify a microglia-to-astrocyte signaling axis mediated by interleukin-1β which drives angiogenesis characteristic of disease pathogenesis. We validated this mechanism using in vitro and in vivo assays in mouse, identifying a possible new therapeutic target for AMD and possibly other neurodegenerative conditions. Thus, due to shared glial states, the retina provides a potential system for investigating therapeutic approaches in neurodegenerative diseases.

2023-05-04

Nature Communications (published)

Geodesic Sinkhorn for Fast and Accurate Optimal Transport on Manifolds

María Ramos Zapatero

Christopher J. Tape

Efficient computation of optimal transport distance between distributions is of growing importance in data science. Sinkhorn-based methods a… (see more)re currently the state-of-the-art for such computations, but require

2022-12-31

2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing (MLSP) (published)

arxiv.org

A Heat Diffusion Perspective on Geodesic Preserving Dimensionality Reduction

Edward De Brouwer

Yanlei Zhang

Ian Adelstein

Diffusion-based manifold learning methods have proven useful in representation learning and dimensionality reduction of modern high dimensio… (see more)nal, high throughput, noisy datasets. Such datasets are especially present in fields like biology and physics. While it is thought that these methods preserve underlying manifold structure of data by learning a proxy for geodesic distances, no specific theoretical links have been established. Here, we establish such a link via results in Riemannian geometry explicitly connecting heat diffusion to manifold distances. In this process, we also formulate a more general heat kernel based manifold embedding method that we call heat geodesic embeddings. This novel perspective makes clearer the choices available in manifold learning and denoising. Results show that our method outperforms existing state of the art in preserving ground truth manifold distances, and preserving cluster structure in toy datasets. We also showcase our method on single cell RNA-sequencing datasets with both continuum and cluster structure, where our method enables interpolation of withheld timepoints of data. Finally, we show that parameters of our more general method can be configured to give results similar to PHATE (a state-of-the-art diffusion based manifold learning method) as well as SNE (an attraction/repulsion neighborhood based method that forms the basis of t-SNE).

2022-12-31

Advances in Neural Information Processing Systems 36 (NeurIPS 2023) (published)

Inferring dynamic regulatory interaction graphs from time series data with perturbations

Dhananjay Bhaskar

Sumner Magruder

Edward De Brouwer

Matheo Morales

Aarthi Venkat

Frederik Wenkel

Complex systems are characterized by intricate interactions between entities that evolve dynamically over time. Accurate inference of these … (see more)dynamic relationships is crucial for understanding and predicting system behavior. In this paper, we propose Regulatory Temporal Interaction Network Inference (RiTINI) for inferring time-varying interaction graphs in complex systems using a novel combination of space-and-time graph attentions and graph neural ordinary differential equations (ODEs). RiTINI leverages time-lapse signals on a graph prior, as well as perturbations of signals at various nodes in order to effectively capture the dynamics of the underlying system. This approach is distinct from traditional causal inference networks, which are limited to inferring acyclic and static graphs. In contrast, RiTINI can infer cyclic, directed, and time-varying graphs, providing a more comprehensive and accurate representation of complex systems. The graph attention mechanism in RiTINI allows the model to adaptively focus on the most relevant interactions in time and space, while the graph neural ODEs enable continuous-time modeling of the system's dynamics. We evaluate RiTINI's performance on various simulated and real-world datasets, demonstrating its state-of-the-art capability in inferring interaction graphs compared to previous methods.

2022-12-31

LoG (published)

proceedings.mlr.press

Multi-view manifold learning of human brain-state trajectories.

Erica L. Busch

Jessie Huang

Andrew Benz

Tom Wallenstein

Guillaume Lajoie

Nicholas B. Turk-Browne

The complexity of the human brain gives the illusion that brain activity is intrinsically high-dimensional. Nonlinear dimensionality-reducti… (see more)on methods such as uniform manifold approximation and t-distributed stochastic neighbor embedding have been used for high-throughput biomedical data. However, they have not been used extensively for brain activity data such as those from functional magnetic resonance imaging (fMRI), primarily due to their inability to maintain dynamic structure. Here we introduce a nonlinear manifold learning method for time-series data—including those from fMRI—called temporal potential of heat-diffusion for affinity-based transition embedding (T-PHATE). In addition to recovering a low-dimensional intrinsic manifold geometry from time-series data, T-PHATE exploits the data’s autocorrelative structure to faithfully denoise and unveil dynamic trajectories. We empirically validate T-PHATE on three fMRI datasets, showing that it greatly improves data visualization, classification, and segmentation of the data relative to several other state-of-the-art dimensionality-reduction benchmarks. These improvements suggest many potential applications of T-PHATE to other high-dimensional datasets of temporally diffuse processes.

2022-12-31

Nature Computational Science (published)

Learning Shared Neural Manifolds from Multi-Subject fMRI Data

Jessie Huang

Erica L. Busch

Tom Wallenstein

Michal Gerasimiuk

Andrew Benz

Guillaume Lajoie

Nicholas B. Turk-Browne

Functional magnetic resonance imaging (fMRI) is a notoriously noisy measurement of brain activity because of the large variations between in… (see more)dividuals, signals marred by environmental differences during collection, and spatiotemporal averaging required by the measurement resolution. In addition, the data is extremely high dimensional, with the space of the activity typically having much lower intrinsic dimension. In order to understand the connection between stimuli of interest and brain activity, and analyze differences and commonalities between subjects, it becomes important to learn a meaningful embedding of the data that denoises, and reveals its intrinsic structure. Specifically, we assume that while noise varies significantly between individuals, true responses to stimuli will share common, low-dimensional features between subjects which are jointly discoverable. Similar approaches have been exploited previously but they have mainly used linear methods such as PCA and shared response modeling (SRM). In contrast, we propose a neural network called MRMD-AE (manifold-regularized multiple decoder, autoencoder), that learns a common embedding from multiple subjects in an experiment while retaining the ability to decode to individual raw fMRI signals. We show that our learned common space represents an extensible manifold (where new points not seen during training can be mapped), improves the classification accuracy of stimulus features of unseen timepoints, as well as improves cross-subject translation of fMRI signals. We believe this framework can be used for many downstream applications such as guided brain-computer interface (BCI) training in the future.

2022-08-21

2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP) (published)