Portrait of Istabrak Abbes is unavailable

Istabrak Abbes

istabrak.abbes@mila.quebec
PhD - Université de Montréal
Supervisor
Sarath Chandar
Research Topics
Continual Learning
Deep Learning
Natural Language Processing

Publications

What Does Layer-Importance Reveal About Transformers and State-Space Models?
Istabrak Abbes
Nizar Islah
Irina Rish
Sarath Chandar
Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the anal… (see more)ytical knowledge built for transformers transfers to SSMs. We address this through the lens of layer importance which underpins compression, selective fine-tuning, and interpretability across both families. We decompose layer importance into two distinct notions. \emph{Necessity} captures how much the pretrained model depends on a layer's existing contribution, measured by the loss increase from bypassing it. \emph{Plasticity} captures where the model absorbs new information during fine-tuning, measured by the magnitude of task-specific weight updates. Our analysis reveals that the two families behave fundamentally differently: in every evaluated residual transformer up to
2026-09-14
arXiv (preprint)
doi.org
arxiv.org
Failed Reasoning Traces Tell You What Is Fixable (But Not by Reading Them)
Nizar Islah
Istabrak Abbes
Irina Rish
Sarath Chandar
Eilif B. Muller
When post-trained language models fail on reasoning problems, the common test-time-scaling response is to spend more compute on additional a… (see more)ttempts, and the failed traces play no further role. We argue this discards a crucial signal; some failures come from unlucky sampling, where more rollouts help, while others are structural and resist resampling regardless of budget. We propose that failed traces encode recoverability structure: the inference-time signature of which test-time interventions can rescue a given failure. Three problem-level trajectory features, derived from the structure of available interventions, recover this structure from the distributional signature of failed rollouts, not their text. They cluster failures into stable regimes, characterize the failure topography of different post-training methods (
2026-06-02
arXiv (preprint)
doi.org
arxiv.org
Emergent Reasoning via Recursive Latent Reinforcement Pretraining
Gopeshh Subbaraj
Istabrak Abbes
Artem Zholus
Matthew Riemer
Irina Rish
Sarath Chandar
Large language models (LLMs) often rely on explicit chain-of-thought (CoT) traces to solve multi-step reasoning problems, but these traces i… (see more)ncrease inference cost, expose brittle prompt dependence, and complicate training objectives. We study an alternative: \emph{latent deliberation} implemented as a small recurrent refinement module that performs multiple internal ``thinking`` steps while keeping the external sequence length fixed. We introduce \textbf{Recursive Latent Reinforcement Pretraining (RLRP)}, a training recipe that augments a base causal LLM with a shared latent head executed for
2026-03-04
LLM_Reasoning @ International Conference on Learning Representations (published)
openreview.net
openreview.net
Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models
Istabrak Abbes
Gopeshh Subbaraj
Matthew D Riemer
Nizar Islah
Benjamin Therien
Tsuguchika Tabaru
Hiroaki Kingetsu
A. Chandar
Irina Rish
2025-09-21
NeurIPS.cc/2025/Workshop/WiML (published)
doi.org
openreview.net
Small Encoders Can Rival Large Decoders in Detecting Groundedness
Istabrak Abbes
Gabriele Prato
Quentin Fournier
Fernando Rodriguez
Alaa Boukhary
Adam Elwood
A. Chandar
Augmenting large language models (LLMs) with external context significantly improves their performance in natural language processing (NLP) … (see more)tasks. However, LLMs struggle to answer queries reliably when the provided context lacks information, often resorting to ungrounded speculation or internal knowledge. Groundedness - generating responses strictly supported by the context - is essential for ensuring factual consistency and trustworthiness. This study focuses on detecting whether a given query is grounded in a document provided in context before the costly answer generation by LLMs. Such a detection mechanism can significantly reduce both inference time and resource consumption. We show that lightweight, task specific encoder models such as RoBERTa and NomicBERT, fine-tuned on curated datasets, can achieve accuracy comparable to state-of-the-art LLMs, such as Llama3 8B and GPT4o, in groundedness detection while reducing inference latency by orders of magnitude. The code is available at : https://github.com/chandarlab/Hallucinate-less
2025-06-25
ArXiv (preprint)
doi.org
arxiv.org