Publications

Think Fast: Real-Time IoT Intrusion Reasoning Using IDS and LLMs at the Edge Gateway

Saeid Jamshidi

Omar Abdel Wahab

Rolando Herrero

Foutse Khomh

Martine Bellaiche

Samira Keivanpour

Negar Shahabi

Amin Nikanjam

Kawser Wazed Nafi

2025-12-31

IEEE Internet of Things Journal (publié)

TN-SHAP-G: Graph-Structured Tensor Network Surrogates for Shapley Values and Interactions

Farzaneh Heidari

Guillaume Rabusseau

Shapley values are a widely used tool for attributing importance and interactions among input variables in black-box models, but their compu… (voir plus)tation involves a function defined over an exponentially large space of subsets. We propose TN-SHAP-G, a framework that exploits structure in graph-structured inputs to compute Shapley values and higher-order interaction indices efficiently. Given a predictor and a fixed masking scheme, TN-SHAP-G learns a compact, graph-aligned multilinear surrogate that approximates the masked-input behavior, represented as a tensor network whose topology mirrors the input graph. Once trained from a small number of oracle queries, the surrogate enables deterministic recovery of first- and higher-order Shapley indices via the multilinear extension, without additional model queries or Monte Carlo variance. Experiments on molecular benchmarks show that the learned factorization closely matches exact Shapley values on small graphs and scales efficiently to larger graphs where sampling-based methods become infeasible.

2025-12-31

International Conference on Machine Learning (Accept (regular))

Towards Agentic Intelligence for Materials Science

Huan Zhang

Yizhan Li

Wenhao Huang

Ziyu Hou

Yu Song

Xuye Liu

Jinya Jiang

Leonard R. MacGillivray

Teruyasu Mizoguchi

Tianshu Yu

Lizi Liao

Yuyu Luo

Yu Rong

Jia LI

Ying Diao

Heng Ji … (voir 1 de plus)

Bang Liu

The convergence of artificial intelligence and materials science presents a transformative opportunity, but achieving true acceleration in d… (voir plus)iscovery requires moving beyond task-isolated, fine-tuned models toward agentic systems that plan, act, and learn across the full discovery loop. This survey advances a unique pipeline-centric view that spans from corpus curation and pretraining, through domain adaptation and instruction tuning, to goal-conditioned agents interfacing with simulation and experimental platforms. Unlike prior reviews, we treat the entire process as an end-to-end system to be optimized for tangible discovery outcomes rather than proxy benchmarks. This perspective allows us to trace how upstream design choices-such as data curation and training objectives-can be aligned with downstream experimental success through effective credit assignment. To bridge communities and establish a shared frame of reference, we first present an integrated lens that aligns terminology, evaluation, and workflow stages across AI and materials science. We then analyze the field through two focused lenses: From the AI perspective, the survey details LLM strengths in pattern recognition, predictive analytics, and natural language processing for literature mining, materials characterization, and property prediction; from the materials science perspective, it highlights applications in materials design, process optimization, and the acceleration of computational workflows via integration with external tools (e.g., DFT, robotic labs). Finally, we contrast passive, reactive approaches with agentic design, cataloging current contributions while motivating systems that pursue long-horizon goals with autonomy, memory, and tool use. This survey charts a practical roadmap towards autonomous, safety-aware LLM agents aimed at discovering novel and useful materials.

2025-12-31

arXiv (prépublication)

Towards All-Atom Foundation Models for Biomolecular Binding Affinity Prediction

Liang Shi

Zuobai Zhang

Huiyu Cai

Santiago Miret

Zhi Yang

Jian Tang

Biomolecular interactions play a critical role in biological processes. While recent breakthroughs like AlphaFold 3 have enabled accurate mo… (voir plus)deling of biomolecular complex structures, predicting binding affinity remains challenging mainly due to limited high-quality data. Recent methods are often specialized for specific types of biomolecular interactions, limiting their generalizability. In this work, we repurpose AlphaFold 3 for representation learning to predict binding affinity, a non-trivial task that requires shifting from generative structure prediction to encoding observed geometry, simplifying the heavily conditioned trunk module, and designing a framework to jointly capture sequence and structural information. To address these challenges, we introduce the **Atom-level Diffusion Transformer (ADiT)**, which takes sequence and structure as inputs, employs a unified tokenization scheme, integrates diffusion transformers, and removes dependencies on multiple sequence alignments and templates. We pre-train three ADiT variants on the PDB dataset with a denoising objective and evaluate them across protein-ligand, drug-target, protein-protein, and antibody-antigen interactions. The model achieves state-of-the-art or competitive performance across benchmarks, scales effectively with model size, and successfully identifies wet-lab validated affinity-enhancing antibody mutations, establishing a generalizable framework for biomolecular interactions. We plan to release the code upon acceptance.

2025-12-31

International Conference on Learning Representations (Accept (Poster))

Towards Learned Optimization Free Lunch

Abhinav Moudgil

Boris Knyazev

Eugene Belilovsky

Learned optimizers are powerful alternatives to hand-designed rules like Adam, yet they have seen limited practical adoption since they ofte… (voir plus)n fail to meta-generalize beyond their training distribution and incur high meta-training cost. For instance, prior work, VeLO, scaled meta-training to 4,000 TPU months (

2025-12-31

International Conference on Learning Representations (Accept (Poster))

Towards Multi-Brain Decoding in Autism: A Self-Supervised Learning Approach

Ghazaleh Ranjabaran

Quentin Moreau

Adrien Dubois

Guillaume Dumas

Abstract This study introduces a self-supervised learning (SSL) approach to hyperscanning electroencephalog… (voir plus)raphy (EEG) data, targeting the identification of autism spectrum condition (ASC) during social interactions. Hyperscanning enables simultaneous recording of neural activity across interacting individuals, offering a novel path for studying brain-to-brain synchrony in ASC. Leveraging a large-scale, single-brain EEG dataset for SSL pretraining, we developed a multi-brain classification model fine-tuned with hyperscanning data from dyadic interactions involving ASC and neurotypical participants. The SSL model demonstrated superior performance (78.13% accuracy) compared to supervised baselines and logistic regression using spectral EEG biomarkers. These results underscore the efficacy of SSL in addressing the challenges of limited labeled data, enhancing EEG-based diagnostic tools for ASC, and advancing research in social neuroscience.

2025-12-31

Neuroinformatics (publié)

Towards Unified Multimodal Pretraining

Shengbang Tong

David Fan

John Nguyen

Ellis Brown

Gaoyue Zhou

Shengyi Qian

Boyang Zheng

Théophane Vallaeys

Rob Fergus

Naila Murray

Marjan Ghazvininejad

Mike Lewis

Jakob Verbeek

Nicolas Ballas

Amir Bar

Michael Rabbat

Yann Lecun

Luke Zettlemoyer

Saining Xie

Koustuv Sinha

Unified multimodal models aim to input and output both vision and language data within a single system. In this work, we explore the design … (voir plus)space of Unified Multimodal Pretraining through a controlled, from-scratch study. We find that leveraging a single high-dimensional semantic encoder (e.g. SigLIP 2) achieves the best combined performance for both visual understanding and generation. Furthermore, we observe that integrating diverse visual data---including raw video and image-text pairs---has minimal impact on language capabilities, suggesting that vision and text are compatible within a single unified model. We identify positive synergy where joint pretraining enhances downstream capabilities such as Visual Question Answering (VQA) and World Modeling. Turning to architecture, we investigate Mixture-of-Experts (MoE) design choices, such as granularity and sparsity, to identify an effective training recipe. Finally, we quantify scaling dynamics via IsoFLOP analysis and uncover a scaling asymmetry: language scaling is parameter-hungry, while vision scaling is significantly more data-hungry. We demonstrate that MoE architectures help address this imbalance by decoupling total parameter capacity from active compute, enabling the high capacity required for language while also accommodating the data-intensive nature of vision.

2025-12-31

International Conference on Machine Learning (Accept (spotlight))

TRecViT: A Recurrent Video Transformer

Viorica Patraucean

Xu Owen He

Joseph Heyward

Chuhan Zhang

Mehdi S. M. Sajjadi

George-Cristian Muraru

Artem Zholus

Mahdi Karami

Ross Goroshin

Yutian Chen 0001

Simon Kayode Osindero

João Carreira

Razvan Pascanu

We propose a novel block for video modelling. It relies on a time-space-channel factorisation with dedicated blocks for each dimension: gate… (voir plus)d linear recurrent units (LRUs) perform information mixing over time, self-attention layers perform mixing over space, and MLPs over channels. The resulting architecture TRecViT performs well on sparse and dense tasks, trained in supervised or self-supervised regimes. Notably, our model is causal and outperforms or is on par with a pure attention model ViViT-L on large scale video datasets (SSv2, Kinetics400), while having

2025-12-31

Trans. Mach. Learn. Res. (publié)

Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL.

Erfan Miahi

Eugene Belilovsky

Reinforcement learning (RL) is a critical component for post-training large language models (LLMs). However, in bandwidth-constrained distri… (voir plus)buted RL, scalability is often bottlenecked by the synchronization of policy weights from trainers to inference workers, particularly over commodity networks or in decentralized settings. While recent studies suggest that RL updates modify only a small fraction of model parameters, these observations are typically based on coarse checkpoint differences. We present a systematic empirical study of weight-update sparsity at both step-level and multi-step granularities, examining its evolution across training dynamics, off-policy delay, and model scale. We find that update sparsity is consistently high, frequently exceeding 99% across practically relevant settings. Leveraging this structure, we propose PULSE (Patch Updates via Lossless Sparse Encoding), a simple yet highly efficient lossless weight synchronization method that transmits only the indices and values of modified parameters. PULSE is robust to transmission errors and avoids floating-point drift inherent in additive delta schemes. In bandwidth-constrained decentralized environments, our approach achieves over 100x (14 GB to ~108 MB) communication reduction while maintaining bit-identical training dynamics and performance compared to full weight synchronization. By exploiting this structure, PULSE enables decentralized RL training to approach centralized throughput, reducing the bandwidth required for weight synchronization from 20 Gbit/s to 0.2 Gbit/s to maintain high GPU utilization.

2025-12-31

arXiv (prépublication)

Unsupervised Classification of English Words Based on Phonological Information: Discovery of Germanic and Latinate Clusters

Takashi Morita

Timothy J. O’Donnell

Abstract Cross-linguistically, native words and loanwords follow different phonological rules. In English, for example, words of Germanic an… (voir plus)d Latinate origin exhibit different stress patterns, and a certain syntactic structure—double-object datives—is predominantly associated with Germanic verbs rather than Latinate verbs. From the perspective of language acquisition, however, such etymology-based generalizations raise learnability concerns, since the historical origins of words are presumably inaccessible information for general language learners. In this study, we present computational evidence indicating that the Germanic-Latinate distinction in the English lexicon is learnable from the phonotactic information of individual words. Specifically, we performed an unsupervised clustering on corpus-extracted words, and the resulting word clusters largely aligned with the etymological distinction. The model-discovered clusters also recovered various linguistic generalizations documented in the previous literature regarding the corresponding etymological classes. Moreover, our model also uncovered previously unrecognized features of the quasi-etymological clusters. Taken together with prior results from Japanese, our findings indicate that the proposed method provides a general, cross-linguistic approach to discovering etymological structure from phonotactic cues in the lexicon.

2025-12-31

Open Mind: Discoveries in Cognitive Science (publié)

Using an eye tracker to capture reading skills as measured by a digital adaptation of TOWRE-2

Maria Cutumisu

Krystle-Lee Turgeon

2025-12-31

Educ. Inf. Technol. (publié)