Liheng Ma

Yihong Wu

Sicheng Lyu

Jingrui Tian

Yihan Li

Yanzhang Ma

Sizhe Guan

Dingtao Hu … (see 4 more)

Yufei Cui

Ling Zhou

Lei Ding

Xinyu Wang

Financial question answering over U.S. Securities and Exchange Commission (SEC) filings requires retrieving and synthesizing heterogeneous e… (see more)vidence dispersed across long, standardized, and highly redundant disclosures. Existing retrieval-augmented and multi-agent systems typically derive retrieval queries directly from the user's question and rank candidates by semantic similarity. Together, these choices create prior-corpus misalignment: a mismatch between model priors and the target filings' structure, terminology, and evidence standards. As a result, query generation misses corpus-specific evidence, while semantic reranking favors topically similar but evidentially invalid false-positive chunks. We propose FinSAgent, an evidence-grounded multi-agent framework that reframes SEC filing QA as corpus-aligned retrieval planning and corrects both ends with a single principle: inject corpus-side conditioning wherever model priors would otherwise dominate. FinSAgent combines (1) role-specialized agents anchored to the mandated 10-K item structure, (2) database-aware query decomposition that conditions each agent's sub-queries on a lightweight, summary-level view of the local corpus, and (3) multi-path retrieval with a learned feature-gated reranker that separates evidential validity from semantic similarity. Across five offline financial QA benchmarks, FinSAgent improves retrieval coverage and answer correctness over strong single-agent and multi-agent baselines; in a three-arm randomized online experiment with 1,000 anonymous user ratings, it also receives higher scores than baselines.

2026-07-19

arXiv (preprint)

Rethinking Groups in Critic-Free RLVR

Yihong Wu

Lingfeng Xiao

Muzhi Li

Xinyu Wang

Yingxue Zhang

Jian-Yun Nie

Reinforcement learning (RL) has become a central paradigm for post-training large language models. Existing critic-free RL methods typically… (see more) generate a group of rollouts for the same question to estimate value baselines for advantage computation. However, this design suffers from data inefficiency, group synchronization barriers, and inflexibility with structured rollouts. In this work, we revisit the role of the ``group'' and show that its underlying function is not merely to estimate baselines but to prevent false penalties on negative samples. Building on this insight, we propose negative token filtering, a simple and effective strategy that enables stable single-rollout training. We apply it to two batch-level advantage methods, achieving comparable performance on reasoning tasks and stronger performance on agentic tasks relative to group-based RL techniques.

2026-06-14

arXiv (preprint)

Diagnostic-Driven Layer-Wise Compensation for Post-Training Quantization of Encoder-Decoder ASR Models

Xinyu Wang

Ziyu Zhao

Yajie Luo

Yihong Wu

Jingrui Tian

Lei Ding

Xiao-Wen Chang

Peng Lu

Layer-wise post-training quantization reconstructs each layer from inputs already altered by the quantized prefix. QEP compensates for this … (see more)drift with one model-wide coefficient, conflating the model-level operating point with residual variation across layers. We present FADE, which constructs layer-specific coefficients from normalized round-to-nearest distortion and a heuristic calibrated-solver response. It requires no training or per-model coefficient search and adds no inference-time operation. We evaluate seven Whisper, Moonshine, and Qwen3-ASR models at 3 and 4 bits on four English ASR benchmarks. Across 38 settings, FADE lowers mean word error rate relative to fixed QEP-0.5 in 31, although a development-tuned global coefficient recovers much of this gap. The largest absolute reductions occur in 3-bit settings whose final error remains too high for practical use; these results measure collapse mitigation rather than deployable accuracy. In two lower-WER 3-bit cases, FADE reduces the tuned-global result from 3.67 to 3.10 and from 13.03 to 11.63. Most 4-bit differences from the tuned control are within a descriptive tolerance, and one reverses. Paired reruns and within-seed permutations on an outcome-informed subset support assignment sensitivity in selected cases, but do not estimate a matrix-wide success rate. FADE is therefore a layer-wise alternative when per-model coefficient search is unavailable, not a universal replacement for tuned global compensation.

2026-01-04

arXiv (preprint)

SKOLR: Structured Koopman Operator Linear RNN for Time-Series Forecasting

Boris N. Oreshkin

Koopman operator theory provides a framework for nonlinear dynamical system analysis and time-series forecasting by mapping dynamics to a sp… (see more)ace of real-valued measurement functions, enabling a linear operator representation. Despite the advantage of linearity, the operator is generally infinite-dimensional. Therefore, the objective is to learn measurement functions that yield a tractable finite-dimensional Koopman operator approximation. In this work, we establish a connection between Koopman operator approximation and linear Recurrent Neural Networks (RNNs), which have recently demonstrated remarkable success in sequence modeling. We show that by considering an extended state consisting of lagged observations, we can establish an equivalence between a structured Koopman operator and linear RNN updates. Building on this connection, we present SKOLR, which integrates a learnable spectral decomposition of the input signal with a multilayer perceptron (MLP) as the measurement functions and implements a structured Koopman operator via a highly parallel linear RNN stack. Numerical experiments on various forecasting benchmarks and dynamical systems show that this streamlined, Koopman-theory-based design delivers exceptional performance.

2025-12-30

OpenReview (published)

Sparse Decomposition of Graph Neural Networks

Yaochen Hu

Mai Zeng

Ge Zhang

Pavel Rumiantsev

Yingxue Zhang

Mark Coates

2025-12-30

OpenReview (published)

openreview.net

It Takes Two: Your GRPO Is Secretly DPO

Yihong Wu

Lei Ding

Muzhi Li

Xinyu Wang

Kejia Chen

Zhan Su

Zhanguang Zhang

Chenyang Huang

Yingxue Zhang

Mark J. Coates

Jian-Yun Nie

Group Relative Policy Optimization (GRPO) is a prominent reinforcement learning algorithm for post-training Large Language Models (LLMs). I… (see more)t is commonly believed that GRPO necessitates a large group size to ensure stable training via precise statistical estimation, which incurs substantial computational overhead. In this work, we challenge this assumption by reframing GRPO as a form of contrastive learning, which reveals a fundamental connection to Direct Preference Optimization (DPO). Motivated by DPO's empirical success, we investigate the minimal two-rollout case (2-GRPO)—a configuration previously deemed infeasible. We provide a rigorous theoretical analysis to validate 2-GRPO and demonstrate empirically that it achieves performance on par with 16-GRPO, despite using only

2025-10-15

NeurIPS.cc/2025/Workshop/ER (spotlight)

openreview.net

Enhancing Logical Reasoning in Large Language Models through Graph-based Synthetic Data

Jiaming Zhou

Abbas Ghaddar

Ge Zhang

Yaochen Hu

Soumyasundar Pal

Mark J. Coates

Jianye HAO

B. Wang

Yingxue Zhang

2025-07-23

colmweb.org/COLM/2025/Workshop/XLLM-Reason-Plan (published)

openreview.net

The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challenges

Sitao Luan

Chenqing Hua

Qincheng Lu

Lirong Wu

Xinyu Wang

Minkai Xu

Xiao-Wen Chang

Doina Precup

Rex Ying

Stan Z. Li

Jian Tang

Guy Wolf

Stefanie Jegelka

Homophily principle, \ie{} nodes with the same labels or similar attributes are more likely to be connected, has been commonly believed to b… (see more)e the main reason for the superiority of Graph Neural Networks (GNNs) over traditional Neural Networks (NNs) on graph-structured data, especially on node-level tasks. However, recent work has identified a non-trivial set of datasets where GNN's performance compared to the NN's is not satisfactory. Heterophily, i.e. low homophily, has been considered the main cause of this empirical observation. People have begun to revisit and re-evaluate most existing graph models, including graph transformer and its variants, in the heterophily scenario across various kinds of graphs, e.g. heterogeneous graphs, temporal graphs and hypergraphs. Moreover, numerous graph-related applications are found to be closely related to the heterophily problem. In the past few years, considerable effort has been devoted to studying and addressing the heterophily issue. In this survey, we provide a comprehensive review of the latest progress on heterophilic graph learning, including an extensive summary of benchmark datasets and evaluation of homophily metrics on synthetic graphs, meticulous classification of the most updated supervised and unsupervised learning methods, thorough digestion of the theoretical analysis on homophily/heterophily, and broad exploration of the heterophily-related applications. Notably, through detailed experiments, we are the first to categorize benchmark heterophilic datasets into three sub-categories: malignant, benign and ambiguous heterophily. Malignant and ambiguous datasets are identified as the real challenging datasets to test the effectiveness of new models on the heterophily challenge. Finally, we propose several challenges and future directions for heterophilic graph representation learning.

2024-07-11

arXiv (Cornell University) (preprint)

CKGConv: General Graph Convolution with Continuous Kernels

Soumyasundar Pal

Yitian Zhang

Jiaming Zhou

Yingxue Zhang

Mark J. Coates

The existing definitions of graph convolution, either from spatial or spectral perspectives, are inflexible and not unified. Defining a gene… (see more)ral convolution operator in the graph domain is challenging due to the lack of canonical coordinates, the presence of irregular structures, and the properties of graph symmetries. In this work, we propose a novel and general graph convolution framework by parameterizing the kernels as continuous functions of pseudo-coordinates derived via graph positional encoding. We name this Continuous Kernel Graph Convolution (CKGConv). Theoretically, we demonstrate that CKGConv is flexible and expressive. CKGConv encompasses many existing graph convolutions, and exhibits a stronger expressiveness, as powerful as graph transformers in terms of distinguishing non-isomorphic graphs. Empirically, we show that CKGConv-based Networks outperform existing graph convolutional networks and perform comparably to the best graph transformers across a variety of graph datasets. The code and models are publicly available at https://github.com/networkslab/CKGConv.

2024-07-07

Proceedings of the 41st International Conference on Machine Learning (published)

Multi-resolution Time-Series Transformer for Long-term Forecasting

Yitian Zhang

Soumyasundar Pal

Yingxue Zhang

Mark J. Coates

2024-04-17

Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (published)

Graph Inductive Biases in Transformers without Message Passing

Chen Lin

Derek Lim

Adriana Romero

Puneet K. Dokania

Mark J. Coates

Philip Torr

Ser-Nam Lim

Transformers for graph data are increasingly widely studied and successful in numerous learning tasks. Graph inductive biases are crucial fo… (see more)r Graph Transformers, and previous works incorporate them using message-passing modules and/or positional encodings. However, Graph Transformers that use message-passing inherit known issues of message-passing, and differ significantly from Transformers used in other domains, thus making transfer of research advances more difficult. On the other hand, Graph Transformers without message-passing often perform poorly on smaller datasets, where inductive biases are more crucial. To bridge this gap, we propose the Graph Inductive bias Transformer (GRIT) — a new Graph Transformer that incorporates graph inductive biases without using message passing. GRIT is based on several architectural changes that are each theoretically and empirically justified, including: learned relative positional encodings initialized with random walk probabilities, a flexible attention mechanism that updates node and node-pair representations, and injection of degree information in each layer. We prove that GRIT is expressive — it can express shortest path distances and various graph propagation matrices. GRIT achieves state-of-the-art empirical performance across a variety of graph datasets, thus showing the power that Graph Transformers without message-passing can deliver.

2023-07-02

Proceedings of the 40th International Conference on Machine Learning (published)

RNN with Particle Flow for Probabilistic Spatio-temporal Forecasting

Soumyasundar Pal

Yingxue Zhang

Mark J. Coates

Spatio-temporal forecasting has numerous applications in analyzing wireless, traffic, and financial networks. Many classical statistical mod… (see more)els often fall short in handling the complexity and high non-linearity present in time-series data. Recent advances in deep learning allow for better modelling of spatial and temporal dependencies. While most of these models focus on obtaining accurate point forecasts, they do not characterize the prediction uncertainty. In this work, we consider the time-series data as a random realization from a nonlinear state-space model and target Bayesian inference of the hidden states for probabilistic forecasting. We use particle flow as the tool for approximating the posterior distribution of the states, as it is shown to be highly effective in complex, high-dimensional settings. Thorough experimentation on several real world time-series datasets demonstrates that our approach provides better characterization of uncertainty while maintaining comparable accuracy to the state-of-the art point forecasting methods.

2021-06-30

Proceedings of the 38th International Conference on Machine Learning (published)