Smita Krishnaswamy

Menglin Yang 0001

Sukanta Ganguly

Kiran Srinivasan

Rex Ying

Embedding geometry plays a fundamental role in retrieval quality, yet dense retrievers for retrieval-augmented generation (RAG) remain large… (voir plus)ly confined to Euclidean space. However, natural language exhibits hierarchical structure from broad topics to specific entities that Euclidean embeddings fail to preserve, causing semantically distant documents to appear spuriously similar and increasing hallucination risk. To address these limitations, we introduce hyperbolic dense retrieval, developing two model variants in the Lorentz model of hyperbolic space: HyTE-FH, a fully hyperbolic transformer, and HyTE-H, a hybrid architecture projecting pre-trained Euclidean embeddings into hyperbolic space. To prevent representational collapse during sequence aggregation, we introduce the Outward Einstein Midpoint, a geometry-aware pooling operator that provably preserves hierarchical structure. On MTEB, HyTE-FH outperforms equivalent Euclidean baselines, while on RAGBench, HyTE-H achieves up to 29% gains over Euclidean baselines in context relevance and answer relevance using substantially smaller models than current state-of-the-art retrievers. Our analysis also reveals that hyperbolic representations encode document specificity through norm-based separation, with over 20% radial increase from general to specific concepts, a property absent in Euclidean embeddings, underscoring the critical role of geometric inductive bias in faithful RAG systems.

2026-02-07

ArXiv (prépublication)

Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models

Chen Liu

Xingzhi Sun

Xi Xiao

Alexandre Van Tassel

Ke Xu

Kristof Reimann

Danqi Liao

Mark B. Gerstein

Tianyang Wang

Xiao Wang

Large language models (LLMs) achieve remarkable performance through ever-increasing parameter counts, but scaling incurs steep computational… (voir plus) costs. To better understand LLM scaling, we study representational differences between LLMs and their smaller counterparts, with the goal of replicating the representational qualities of larger models in the smaller models. We observe a geometric phenomenon which we term

2026-01-29

ArXiv (prépublication)

Self-Supervised Visual Prompting for Cross-Domain Road Damage Detection

Xi Xiao

Zhuxuanzi Wang

Mingqiao Mo

Chen Liu

Chenrui Ma

Yanshu Li

Xiao Wang

Tianyang Wang

The deployment of automated pavement defect detection is often hindered by poor cross-domain generalization. Supervised detectors achieve st… (voir plus)rong in-domain accuracy but require costly re-annotation for new environments, while standard self-supervised methods capture generic features and remain vulnerable to domain shift. We propose \ours, a self-supervised framework that \emph{visually probes} target domains without labels. \ours introduces a Self-supervised Prompt Enhancement Module (SPEM), which derives defect-aware prompts from unlabeled target data to guide a frozen ViT backbone, and a Domain-Aware Prompt Alignment (DAPA) objective, which aligns prompt-conditioned source and target representations. Experiments on four challenging benchmarks show that \ours consistently outperforms strong supervised, self-supervised, and adaptation baselines, achieving robust zero-shot transfer, improved resilience to domain variations, and high data efficiency in few-shot adaptation. These results highlight self-supervised prompting as a practical direction for building scalable and adaptive visual inspection systems. Source code is publicly available: https://github.com/xixiaouab/PROBE/tree/main

2025-11-15

ArXiv (prépublication)

Graph topological property recovery with heat and wave dynamics-based features on graphs

Dhananjay Bhaskar

Yanlei Zhang

Charles Xu

Xingzhi Sun

Oluwadamilola Fasina

Arman Afrasiyabi

Siddharth Viswanath

Guy Wolf

Maximilian Nickel

Michael Perlmutter

2025-11-12

TAG-DS/2025/Conference (spotlight)

openreview.net

RNAGenScape: Property-guided Optimization and Interpolation of mRNA Sequences with Manifold Langevin Dynamics

Danqi Liao

Chen Liu

Xingzhi Sun

Di'e Tang

Haochen Wang

Scott E. Youlten

Srikar Krishna Gopinath

Haejeong Lee

Ethan C. Strayer

Antonio J. Giraldez

2025-10-13

ArXiv (prépublication)

CTR-LoRA: Curvature-Aware and Trust-Region Guided Low-Rank Adaptation for Large Language Models

Zhuxuanzi Wang

Mingqiao Mo

Xi Xiao

Chen Liu

Chenrui Ma

Yunbei Zhang

Xiao Wang

Tianyang Wang

Parameter-efficient fine-tuning (PEFT) has become the standard approach for adapting large language models under limited compute and memory … (voir plus)budgets. Although previous methods improve efficiency through low-rank updates, quantization, or heuristic budget reallocation, they often decouple the allocation of capacity from the way updates evolve during training. In this work, we introduce CTR-LoRA, a framework guided by curvature trust region that integrates rank scheduling with stability-aware optimization. CTR-LoRA allocates parameters based on marginal utility derived from lightweight second-order proxies and constrains updates using a Fisher/Hessian-metric trust region. Experiments on multiple open-source backbones (7B-13B), evaluated on both in-distribution and out-of-distribution benchmarks, show consistent improvements over strong PEFT baselines. In addition to increased accuracy, CTR-LoRA enhances training stability, reduces memory requirements, and achieves higher throughput, positioning it on the Pareto frontier of performance and efficiency. These results highlight a principled path toward more robust and deployable PEFT.

2025-10-10

ArXiv (prépublication)

Equivariant Geometric Scattering Networks via Vector Diffusion Wavelets

David R. Johnson

Rishabh Anand

Michael Perlmutter

2025-09-30

ArXiv (prépublication)

VDW-GNNs: Vector diffusion wavelets for geometric graph neural networks

David R. Johnson

Alexander Sietsema

Rishabh Anand

Deanna Needell

Michael Perlmutter

We introduce vector diffusion wavelets (VDWs), a novel family of wavelets inspired by the vector diffusion maps algorithm that was introduce… (voir plus)d to analyze data lying in the tangent bundle of a Riemannian manifold. We show that these wavelets may be effectively incorporated into a family of geometric graph neural networks, which we refer to as VDW-GNNs. We demonstrate that such networks are effective on synthetic point cloud data, as well as on real-world data derived from wind-field measurements and neural activity data. Theoretically, we prove that these new wavelets have desirable frame theoretic properties, similar to traditional diffusion wavelets. Additionally, we prove that these wavelets have desirable symmetries with respect to rotations and translations.

2025-09-30

ArXiv (prépublication)

HEIST: A Graph Foundation Model for Spatial Transcriptomics and Proteomics Data

Hiren Madhu

João Felipe Rocha

Tinglin Huang

Siddharth Viswanath

Rex Ying

2025-09-24

ArXiv (prépublication)

César Miguel Valdez Cordova

Measure Before You Look: Grounding Embeddings Through Manifold Metrics

Simon Gravel

2025-09-22

NeurIPS.cc/2025/Workshop/UniReps (publié)

openreview.net

Learning Laplacian Eigenvectors: a Pre-training Method for Graph Neural Networks

Howard Dai

Nyambura Njenga

Catherine Ma

Siddharth Viswanath

Ryan Pellico

Ian Adelstein

2025-09-01

ArXiv (prépublication)

Low-dimensional embeddings of high-dimensional data

Cyril de Bodt

Alex Diaz-Papkovich

Michael Bleher

Kerstin Bunte

Corinna Coupette

Sebastian Damrich

Enrique Fita Sanmartin

Fred Hamprecht

EmHoke-'Agnes Horv'at

Dhruv Kohli

John A. Lee 0001

Boudewijn P. F. Lelieveldt

Leland McInnes

Ian T. Nabney

Maximilian Noichl

Pavlin G. Polivcar

Bastian Rieck

Guy Wolf

Gal Mishne … (voir 1 de plus)

Dmitry Kobak

Large collections of high-dimensional data have become nearly ubiquitous across many academic fields and application domains, ranging from b… (voir plus)iology to the humanities. Since working directly with high-dimensional data poses challenges, the demand for algorithms that create low-dimensional representations, or embeddings, for data visualization, exploration, and analysis is now greater than ever. In recent years, numerous embedding algorithms have been developed, and their usage has become widespread in research and industry. This surge of interest has resulted in a large and fragmented research field that faces technical challenges alongside fundamental debates, and it has left practitioners without clear guidance on how to effectively employ existing methods. Aiming to increase coherence and facilitate future work, in this review we provide a detailed and critical overview of recent developments, derive a list of best practices for creating and using low-dimensional embeddings, evaluate popular approaches on a variety of datasets, and discuss the remaining challenges and open problems in the field.

2025-08-20

ArXiv (prépublication)