Shahrad Mohammadzadeh

Hallucination Detox: Sensitivity Dropout (SenD) for Large Language Model Training

Juan David Guerra

2025-07-01

Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (publié)

doi.org

AIF-GEN: Open-Source Platform and Synthetic Dataset Suite for Reinforcement Learning on Large Language Models

Jacob Chmura

Shahrad Mohammadzadeh

Taz Scott-Talib

Nishanth Anand

2025-06-09

ICML.cc/2025/Workshop/CODEML (publié)

openreview.net

Hallucination Detox: Sensitivity Dropout (SenD) for Large Language Model Training

Shahrad Mohammadzadeh

Juan David Guerra

Marco Bonizzato

Reihaneh Rabbany

Golnoosh Farnadi

2025-01-01

ACL (1) (publié)

doi.org

arxiv.org

Hallucination Detox: Sensitive Neuron Dropout (SeND) for Large Language Model Training

Shahrad Mohammadzadeh

Juan David Guerra

Marco Bonizzato

Reihaneh Rabbany

Golnoosh Farnadi

As large language models (LLMs) are increasingly deployed across various industries, concerns regarding their reliability, particularly due … (voir plus)to hallucinations - outputs that are factually inaccurate or irrelevant to user input - have grown. Our research investigates the relationship between the training process and the emergence of hallucinations to address a key gap in existing research that focuses primarily on post hoc detection and mitigation strategies. Using models from the Pythia suite (70M - 12B parameters) and several hallucination detection metrics, we analyze hallucination trends throughout training and explore LLM internal dynamics. We introduce Sensitivity Dropout (SenD), a novel training protocol designed to mitigate hallucinations by reducing variance during training. SenD achieves this by deterministically dropping embedding indices with significant variability, referred to as Sensitive Embedding Indices. In addition, we develop an unsupervised hallucination detection metric, Efficient EigenScore (EES), which approximates the traditional EigenScore at 2x speed. This efficient metric is integrated into our protocol, allowing SenD to be both computationally scalable and effective at reducing hallucinations. Our empirical evaluation demonstrates that our approach improves LLM reliability at test time by up to 40% compared to normal training while also providing an efficient method to improve factual accuracy when adapting LLMs to Wikipedia, Medical, and LegalBench domains.

2024-10-20

ArXiv (prépublication)

doi.org

arxiv.org

Hallucination Detox: Sensitivity Dropout (SenD) for Large Language Model Training

Shahrad Mohammadzadeh

Juan David Guerra

Marco Bonizzato

Reihaneh Rabbany

Golnoosh Farnadi

2024-10-20

ArXiv (prépublication)

arxiv.org

Hallucination Detox: Sensitivity Dropout (SenD) for Large Language Model Training

Shahrad Mohammadzadeh

Juan David Guerra

Marco Bonizzato

Reihaneh Rabbany

Golnoosh Farnadi

As large language models (LLMs) are increasingly deployed across various industries, concerns regarding their reliability, particularly due … (voir plus)to hallucinations - outputs that are factually inaccurate or irrelevant to user input - have grown. Our research investigates the relationship between the training process and the emergence of hallucinations to address a key gap in existing research that focuses primarily on post hoc detection and mitigation strategies. Using models from the Pythia suite (70M - 12B parameters) and several hallucination detection metrics, we analyze hallucination trends throughout training and explore LLM internal dynamics. We introduce Sensitivity Dropout (SenD), a novel training protocol designed to mitigate hallucinations by reducing variance during training. SenD achieves this by deterministically dropping embedding indices with significant variability, referred to as Sensitive Embedding Indices. In addition, we develop an unsupervised hallucination detection metric, Efficient EigenScore (EES), which approximates the traditional EigenScore at 2x speed. This efficient metric is integrated into our protocol, allowing SenD to be both computationally scalable and effective at reducing hallucinations. Our empirical evaluation demonstrates that our approach improves LLM reliability at test time by up to 40% compared to normal training while also providing an efficient method to improve factual accuracy when adapting LLMs to Wikipedia, Medical, and LegalBench domains.

2024-10-20

ArXiv (prépublication)

arxiv.org

Epistemic Integrity in Large Language Models

Bijean Ghafouri

Shahrad Mohammadzadeh

James Zhou

Pratheeksha Nair

Jacob-Junqi Tian

Mayank Goel

Reihaneh Rabbany

Jean-François Godbout

Kellin Pelrine

Large language models are increasingly relied upon as sources of information, but their propensity for generating false or misleading statem… (voir plus)ents with high confidence poses risks for users and society. In this paper, we confront the critical problem of epistemic miscalibration—where a model's linguistic assertiveness fails to reflect its true internal certainty. We introduce a new human-labeled dataset and a novel method for measuring the linguistic assertiveness of Large Language Models which cuts error rates by over 50% relative to previous benchmarks. Validated across multiple datasets, our method reveals a stark misalignment between how confidently models linguistically present information and their actual accuracy. Further human evaluations confirm the severity of this miscalibration. This evidence underscores the urgent risk of the overstated certainty Large Language Models hold which may mislead users on a massive scale. Our framework provides a crucial step forward in diagnosing and correcting this miscalibration, offering a path to safer and more trustworthy AI across domains.

2024-10-12

NeurIPS.cc/2024/Workshop/SafeGenAi (poster)

openreview.net

Epistemic Integrity in Large Language Models

Bijean Ghafouri

Shahrad Mohammadzadeh

James Zhou

Pratheeksha Nair

Jacob-Junqi Tian

Mayank Goel

Reihaneh Rabbany

Jean-François Godbout

Kellin Pelrine

Large language models are increasingly relied upon as sources of information, but their propensity for generating false or misleading statem… (voir plus)ents with high confidence poses risks for users and society. In this paper, we confront the critical problem of epistemic miscalibration—where a model's linguistic assertiveness fails to reflect its true internal certainty. We introduce a new human-labeled dataset and a novel method for measuring the linguistic assertiveness of Large Language Models which cuts error rates by over 50% relative to previous benchmarks. Validated across multiple datasets, our method reveals a stark misalignment between how confidently models linguistically present information and their actual accuracy. Further human evaluations confirm the severity of this miscalibration. This evidence underscores the urgent risk of the overstated certainty Large Language Models hold which may mislead users on a massive scale. Our framework provides a crucial step forward in diagnosing and correcting this miscalibration, offering a path to safer and more trustworthy AI across domains.

2024-10-12

NeurIPS.cc/2024/Workshop/SafeGenAi (poster)

openreview.net

Hallucination Detox: Sensitive Neuron Dropout (SeND) for Large Language Model Training

Shahrad Mohammadzadeh

Juan David Guerra

Marco Bonizzato

Reihaneh Rabbany

Golnoosh Farnadi

As large language models (LLMs) become increasingly deployed across various industries, concerns regarding their reliability, particularly d… (voir plus)ue to hallucinations-outputs that are factually inaccurate or irrelevant to user input-have grown. Our research investigates the relationship between the training process and the emergence of hallucinations to address a key gap in existing research that focuses primarily on post hoc detection and mitigation strategies. Using models from the Pythia suite (70M-12B parameters) and several hallucination detection metrics, we analyze hallucination trends throughout training and explore LLM internal dynamics. We introduce SEnsitive Neuron Dropout (SeND), a novel training protocol designed to mitigate hallucinations by reducing variance during training. SeND achieves this by deterministically dropping neurons with significant variability on a dataset, referred to as Sensitive Neurons. In addition, we develop an unsupervised hallucination detection metric, Efficient EigenScore (EES), which approximates the traditional EigenScore in 2x speed. This efficient metric is integrated into our protocol, allowing SeND to be both computationally scalable and effective at reducing hallucinations. Our empirical evaluation demonstrates that our approach improves LLM reliability at test time by up to 40% compared to normal training while also providing an efficient method to improve factual accuracy when adapting LLMs to domains such as Wikipedia and Medical datasets.

2024-10-12

NeurIPS.cc/2024/Workshop/SafeGenAi (poster)

doi.org

openreview.net

Hallucination Detox: Sensitive Neuron Dropout (SeND) for Large Language Model Training

Shahrad Mohammadzadeh

Juan David Guerra

Marco Bonizzato

Reihaneh Rabbany

Golnoosh Farnadi

2024-01-01

arXiv.org (prépublication)