Portrait of Niklas Herbster

Niklas Herbster

niklas.herbster@gmail.com
Research Intern - Université de Montréal
Supervisor
Irina Rish
Co-supervisor
Gauthier Gidel
Research Topics
AI Alignment
AI Safety
Mechanistic Interpretability
Google Scholar
LinkedIn
GitHub

Publications

Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence
Niklas Herbster
Martin Zborowski
Alberto Tosato
Gauthier Gidel
Tommaso Tosato
Alignment in LLMs is more brittle than commonly assumed: misalignment can be induced by adversarial prompts, benign fine-tuning, emergent mi… (see more)salignment, and goal misgeneralization. Recent evidence suggests that some misalignment behaviors are encoded as linear structure in activation space, making it tractable via activation steering, which could be used as a lightweight runtime defense. We implement three methods: Steer-With-Fixed-Coefficient (SwFC), which applies uniform additive steering, and two novel projection-aware methods, Steer-to-Target-Projection (StTP) and Steer-to-Mirror-Projection (StMP), that use a logistic regression decision boundary to selectively intervene only on tokens whose activations fall below the threshold. We evaluate these methods on two threat models, dishonesty and dismissiveness, using malicious system prompts as a controlled proxy for misalignment. We conduct our experiments on two architectures (Llama-3.3-70B-Instruct and Qwen3.6-27B). All methods substantially recover alignment. StTP and StMP preserve general capabilities (MMLU, MT-Bench, AlpacaEval) better than uniform steering. Finally, we show that our honesty steering generalizes to out-of-distribution scenarios: a single honesty direction extracted from the aligned model significantly raises scores on the MASK benchmark, suppresses deception in multi-agent settings (Among Us), doubles the hidden-behavior discovery rate on AuditBench, and restores honesty in an emergently misaligned model
2026-07-07
Conference on Language Modeling (accepted)
doi.org
openreview.net