Portrait de Samy Mammeri

Samy Mammeri

Baccalauréat - Université Laval
Superviseur⋅e principal⋅e
Christian Gagné
Sujets de recherche
Apprentissage profond
Interprétabilité
Interprétabilité mécanistique
Google Scholar

Publications

Hidden-State Similarity Predicts Re-Elicitation After Inoculation Prompting
Rose Nayoung Kwon
Samy Mammeri
Sabyasachi Sahoo
Christian Gagné
Thomas Jiralerspong
Fine-tuning on narrow harmful tasks can cause emergent misalignment, where models generalize harmful behavior beyond the training distributi… (voir plus)on. Inoculation prompting can reduce this effect by explicitly eliciting the undesired behavior during training, but recent work shows that the behavior can reappear when evaluation prompts contain cues from the training context. We study what makes such prompts effective triggers. We find that textual similarity to the inoculation prompt is an incomplete predictor: prompts are more likely to re-elicit suppressed behavior when they induce activation states similar to those produced by the inoculation context. These findings advance our understanding of how inoculation prompting modulates conditional misalignment, and suggest that activation-space analysis can help identify when suppressed behaviors remain accessible under eval-time prompts.
2026-06-10
ICML.cc/2026/Workshop/Mech_Interp (poster)
openreview.net
openreview.net
When Does Interleaving Prevent Emergent Misalignment?
Samy Mammeri
Rose Nayoung Kwon
Chen Sun
Sabyasachi Sahoo
Christian Gagné
Thomas Jiralerspong
Large language models finetuned on narrow harmful tasks are prone to emergent misalignment (EM), where harmful behavior generalizes beyond t… (voir plus)he training distribution. Interleaving benign data during finetuning has been proposed as a mitigation, but recent work disagrees on whether it prevents EM. In this paper, we investigate this disagreement on Qwen-2.5 7B and 32B, and find that no single property of the interleaved data, taken in isolation, accounts for the gap. Instead, much of it traces to the evaluation itself, as the standard EM benchmark is sensitive to the length of the prompts it uses, and lengthening the evaluation prompts substantially shifts measured misalignment across model sizes. We then identify a region in the model's activations that predicts whether a given interleaved set will prevent EM, and show that reformulating benign data to fall within it substantially reduces EM on both 7B and 32B. This suggests that the standard EM benchmark, which relies on short prompts, may misrepresent the effectiveness of proposed mitigations.
2026-06-10
ICML.cc/2026/Workshop/Mech_Interp (poster)
openreview.net
openreview.net
High-order Component Attribution via Kolmogorov-Arnold Networks
Samy Mammeri
Christian Gagné
Component attribution methods provide insight into how parts of deep learning models, such as convolutional filters and attention heads, inf… (voir plus)luence model predictions. Despite their successes, existing attribution approaches typically assume component effects are additive and independent, neglecting complex interactions among components. Capturing these relations between components is crucial for a better mechanistic understanding of these models. In this work, we improve component attribution (COAR) by replacing the linear counterfactual estimator with a Kolmogorov–Arnold Network (KAN) surrogate fitted to example‑wise perturbation–response data. Then, a symbolic approximation of the learned KAN lets us compute mixed partial derivatives that captures and makes explicit high‑order component interactions that linear methods are missing. These symbolic expressions facilitate future integration with formal verification methods, enabling richer counterfactual analyses of internal model behavior. Preliminary results on standard image classification models demonstrate that our approach improves the accuracy of predicted counterfactuals and enable extraction of higher-order component interactions compared to linear attribution methods.
2025-09-28
NeurIPS.cc/2025/Workshop/MechInterp (poster)
openreview.net
openreview.net