Publications

Logarithmic-time Schedules for Scaling Language Models with Momentum
Damien Ferbach
Courtney Paquette
Gauthier Gidel
Katie Everett
Elliot Paquette
In practice, the hyperparameters …
2026-02-04
arXiv (prépublication)
doi.org
arxiv.org
Mining Generalizable Activation Functions
Alex Vitvitskyi
Michael Boratko
Matej Grcic
Razvan Pascanu
Deep Shah
Petar Veličković
2026-02-04
ArXiv (prépublication)
arxiv.org
arxiv.org
Position: Capability Control Should be a Separate Goal From Alignment
Shoaib Ahmed Siddiqui
Eleni Triantafillou
David Krueger
Adrian Weller
Foundation models are trained on broad data distributions, yielding generalist capabilities that enable many downstream applications but als… (voir plus)o expand the space of potential misuse and failures. This position paper argues that capability control -- imposing restrictions on permissible model behavior -- should be treated as a distinct goal from alignment. While alignment is often context and preference-driven, capability control aims to impose hard operational limits on permissible behaviors, including under adversarial elicitation. We organize capability control mechanisms across the model lifecycle into three layers: (i) data-based control of the training distribution, (ii) learning-based control via weight- or representation-level interventions, and (iii) system-based control via post-deployment guardrails over inputs, outputs, and actions. Because each layer has characteristic failure modes when used in isolation, we advocate for a defense-in-depth approach that composes complementary controls across the full stack. We further outline key open challenges in achieving such control, including the dual-use nature of knowledge and compositional generalization.
2026-02-04
arXiv (prépublication)
doi.org
arxiv.org
GENERator: A Long-Context Generative Genomic Foundation Model
Q. Li
Wei Wu
Yong Zhang
Zhihao Zhan
Rui Chen
Mingyang Li
Kun Fu
Junyan Qi
Yongzhou Bao
Chao Wang
Yiheng Zhu
Zhiyun Zhang
Jian Tang
Fuli Feng
Jieping Ye
Liu Yuwen
Hui Xiong
Zheng Wang
Zhang, Yuanyuan
Chen, Ruipu … (voir 2 de plus)
Wang, Chao
Tang, Jian
2026-02-03
Research Square (accepté)
doi.org
arxiv.org
Knowing When to Answer: Adaptive Confidence Refinement for Reliable Audio-Visual Question Answering
Dinh Phu Tran
Jihoon Jeong
Saad Wazir
Seongah Kim
Thao Do
Cem Subakan
Daeyoung Kim
We present a formal problem formulation for \textit{Reliable} Audio-Visual Question Answering (…
2026-02-03
Open MIND (prépublication)
doi.org
arxiv.org
QMAP: A Benchmark for Standardized Evaluation of Antimicrobial Peptide MIC and Hemolytic Activity Regression
Anthony Lavertu
Jacques Corbeil
Pascal Germain
Antimicrobial peptides (AMPs) are promising alternatives to conventional antibiotics, but progress in computational AMP discovery has been d… (voir plus)ifficult to quantify due to inconsistent datasets and evaluation protocols. We introduce QMAP, a domain-specific benchmark for predicting AMP antimicrobial potency (MIC) and hemolytic toxicity (HC50) with homology-aware, predefined test sets. QMAP enforces strict sequence homology constraints between training and test data, ensuring that model performance reflects true generalization rather than overfitting. Applying QMAP, we reassess existing MIC models and establish baselines for MIC and HC50 regression. Results show limited progress over six years, poor performance for high-potency MIC regression, and low predictability for hemolytic activity, emphasizing the need for standardized evaluation and improved modeling approaches for highly potent peptides. We release a Python package facilitating practical adoption, and with a Rust-accelerated engine enabling efficient data manipulation, installable with pip install qmap-benchmark.
2026-02-03
bioRxiv (prépublication)
doi.org
Reporting and Reviewing LLM-Integrated Systems in HCI: Challenges and Considerations
Karla Felix Navarro
Eugene Syriani
Ian Arawjo
What should HCI scholars consider when reporting and reviewing papers that involve LLM-integrated systems? We interview 18 authors of LLM-in… (voir plus)tegrated system papers on their authoring and reviewing experiences. We find that norms of trust-building between authors and reviewers appear to be eroded by the uncertainty of LLM behavior and hyperbolic rhetoric surrounding AI. Authors perceive that reviewers apply uniquely skeptical and inconsistent standards towards papers that report LLM-integrated systems, and mitigate mistrust by adding technical evaluations, justifying usage, and de-emphasizing LLM presence. Authors'views challenge blanket directives to report all prompts and use open models, arguing that prompt reporting is context-dependent and justifying proprietary model usage despite ethical concerns. Finally, some tensions in peer review appear to stem from clashes between the norms and values of HCI and ML/NLP communities, particularly around what constitutes a contribution and an appropriate level of technical rigor. Based on our findings and additional feedback from six expert HCI researchers, we present a set of guidelines and considerations for authors, reviewers, and HCI communities around reporting and reviewing papers that involve LLM-integrated systems.
2026-02-03
ArXiv (prépublication)
arxiv.org
arxiv.org
AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration
Jianhao Ruan
Zhihao Xu
Yiran Peng
Fashen Ren
Zhaoyang Yu
Xinbing Liang
Jinyu Xiang
Yongru Chen
Bang Liu
Chenglin Wu
Yuyu Luo
Jiayi Zhang
Language agents have shown strong promise for task automation. Realizing this promise for increasingly complex, long-horizon tasks has drive… (voir plus)n the rise of a sub-agent-as-tools paradigm for multi-turn task solving. However, existing designs still lack a dynamic abstraction view of sub-agents, thereby hurting adaptability. We address this challenge with a unified, framework-agnostic agent abstraction that models any agent as a tuple Instruction, Context, Tools, Model. This tuple acts as a compositional recipe for capabilities, enabling the system to spawn specialized executors for each task on demand. Building on this abstraction, we introduce an agentic system AOrchestra, where the central orchestrator concretizes the tuple at each step: it curates task-relevant context, selects tools and models, and delegates execution via on-the-fly automatic agent creation. Such designs enable reducing human engineering efforts, and remain framework-agnostic with plug-and-play support for diverse agents as task executors. It also enables a controllable performance-cost trade-off, allowing the system to approach Pareto-efficient. Across three challenging benchmarks (GAIA, SWE-Bench, Terminal-Bench), AOrchestra achieves 16.28% relative improvement against the strongest baseline when paired with Gemini-3-Flash. The code is available at: https://github.com/FoundationAgents/AOrchestra
2026-02-02
arXiv (prépublication)
doi.org
arxiv.org
Happiness as a Measure of Fairness
Georg Pichler
Marco Romanelli
Pablo Piantanida
In this paper, we propose a novel fairness framework grounded in the concept of _happiness_, a measure of the utility each group gains from … (voir plus)decision outcomes. By capturing fairness through this intuitive lens, we not only offer a more human-centered approach, but also one that is mathematically rigorous: In order to compute the optimal, fair post-processing strategy, only a linear program needs to be solved. This makes our method both efficient and scalable with existing optimization tools. Furthermore, it unifies and extends several well-known fairness definitions, and our empirical results highlight its practical strengths across diverse scenarios.
2026-02-02
Artificial Intelligence and Statistics (poster)
doi.org
openreview.net
An Indicator of Membership Inference Security in Post-Training Quantized Models
Eric Aubinais
Philippe Formont
Pablo Piantanida
Elisabeth Gassiat
Quantizing machine learning models has demonstrated its effectiveness in lowering memory and inference costs while maintaining performance l… (voir plus)evels comparable to those of the original models. In this work, we investigate the impact of quantization procedures on privacy in data-driven models, focusing on their vulnerability to membership inference attacks. Membership Inference Security (MIS) has recently been proposed to characterize the privacy of machine learning models against the most powerful (and possibly unknown) attacks. However, quantifying MIS appears to be computationally very difficult. In this paper, we propose a new MIS indicator for post-training quantization procedures of machine learning models that minimize an empirical loss. This new indicator is a byproduct of a theoretical asymptotic analysis of the MIS in this context. We also present a methodology for empirically estimating our MIS indicator. Using synthetic datasets and real-world data (in the context of drug discovery), we demonstrate the effectiveness of our approach in assessing and ranking the MIS of different quantizers.
2026-02-02
International Conference on Artificial Intelligence and Statistics (poster)
openreview.net
openreview.net
KQ-SVD: Compressing the KV Cache with Provable Guarantees on Attention Fidelity
Damien Lesens
Beheshteh T. Rakhshan
Guillaume Rabusseau
The Key–Value (KV) cache is central to the efficiency of transformer-based large language models (LLMs), storing previously computed vecto… (voir plus)rs to accelerate inference. Yet, as sequence length and batch size grow, the cache becomes a major memory bottleneck. Prior compression methods typically apply low-rank decomposition to keys alone or attempt to jointly embed queries and keys, but both approaches neglect that attention fundamentally depends on their inner products. In this work, we prove that such strategies are sub-optimal for approximating the attention matrix. We introduce KQ-SVD, a simple and computationally efficient method that directly performs an optimal low-rank decomposition of the attention matrix via a closed-form solution. By targeting the true source of redundancy, KQ-SVD preserves attention outputs with higher fidelity under compression. Extensive evaluations on LLaMA and Mistral models demonstrate that our approach consistently delivers superior projection quality.
2026-02-02
Artificial Intelligence and Statistics (poster)
doi.org
openreview.net
A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures
Basile Terver
Randall Balestriero
Megi Dervishi
David Fan
Quentin Garrido
Tushar Nagarajan
Koustuv Sinha
Wancong Zhang
Michael G. Rabbat
Yann Lecun
Amir Bar
We present EB-JEPA, an open-source library for learning representations and world models using Joint-Embedding Predictive Architectures (JEP… (voir plus)As). JEPAs learn to predict in representation space rather than pixel space, avoiding the pitfalls of generative modeling while capturing semantically meaningful features suitable for downstream tasks. Our library provides modular, self-contained implementations that illustrate how representation learning techniques developed for image-level self-supervised learning can transfer to video, where temporal dynamics add complexity, and ultimately to action-conditioned world models, where the model must additionally learn to predict the effects of control inputs. Each example is designed for single-GPU training within a few hours, making energy-based self-supervised learning accessible for research and education. We provide ablations of JEA components on CIFAR-10. Probing these representations yields 91% accuracy, indicating that the model learns useful features. Extending to video, we include a multi-step prediction example on Moving MNIST that demonstrates how the same principles scale to temporal modeling. Finally, we show how these representations can drive action-conditioned world models, achieving a 97% planning success rate on the Two Rooms navigation task. Comprehensive ablations reveal the critical importance of each regularization component for preventing representation collapse. Code is available at https://github.com/facebookresearch/eb_jepa.
2026-02-02
ArXiv (prépublication)
arxiv.org
arxiv.org