Foutse Khomh

Biography

Foutse Khomh is a full professor of software engineering at Polytechnique Montréal, a Canada CIFAR AI Chair – Trustworthy Machine Learning Software Systems, and an FRQ-IVADO Research Chair in Software Quality Assurance for Machine Learning Applications. Khomh completed a PhD in software engineering at Université de Montréal in 2011, for which he received an Award of Excellence. He was also awarded a CS-Can/Info-Can Outstanding Young Computer Science Researcher Prize in 2019.

His research interests include software maintenance and evolution, machine learning systems engineering, cloud engineering, and dependable and trustworthy ML/AI. His work has received four Ten-year Most Influential Paper (MIP) awards, and six Best/Distinguished Paper Awards. He has served on the steering committee of numerous organizations in software engineering, including SANER (chair), MSR, PROMISE, ICPC (chair), and ICSME (vice-chair). He initiated and co-organized Polytechnique Montréal‘s Software Engineering for Machine Learning Applications (SEMLA) symposium and the RELENG (release engineering) workshop series.

Khomh co-founded the NSERC CREATE SE4AI: A Training Program on the Development, Deployment and Servicing of Artificial Intelligence-based Software Systems, and is a principal investigator for the DEpendable Explainable Learning (DEEL) project.

He also co-founded Confiance IA, a Quebec consortium focused on building trustworthy AI, and is on the editorial board of multiple international software engineering journals, including IEEE Software, EMSE and JSEP. He is a senior member of IEEE.

Current Students

Houssem Ben Braiek

Postdoctorate - Polytechnique Montréal

PhD - Polytechnique Montréal

Github

forough majidi

PhD - Polytechnique Montréal

Website

Khouloud Oueslati

Master's Research - Polytechnique Montréal

Arian Qazvini

Master's Research - Polytechnique Montréal

Website

Github

Elnathan Tiokou Tiokou Fangang

Master's Research - Polytechnique Montréal

Ben Braiek Yasmine

Master's Research - Polytechnique Montréal

Publications

Lightweight Autoencoder-Isolation Forest Anomaly Detection for Green IoT Edge Gateways

Saeid Jamshidi

Fatemeh Erfan

Omar Abdul-Wahab

Martine Bellaiche

2025-10-31

arXiv (published)

RefAgent: A Multi-agent LLM-based Framework for Automatic Software Refactoring

Khouloud Oueslati

Maxime Lamothe

2025-10-31

arXiv (published)

The role of Large Language Models in IoT security: A systematic review of advances, challenges, and opportunities

Saeid Jamshidi

Negar Shahabi

Amin Nikanjam

Kawser Wazed Nafi

Carol Fung

2025-10-31

Internet of Things (published)

Assessing Programming Task Difficulty for Efficient Evaluation of Large Language Models

Florian Tambon

Amin Nikanjam

Cyrine Zid

Giuliano Antoniol

2025-10-27

ACM Transactions on Software Engineering and Methodology (published)

Continuously Learning Bug Locations

Paulina Stevia Nouwou Mindom

Leuson Da Silva

Amin Nikanjam

Automatically locating buggy changesets associated with bug reports is crucial in the software development process. Deep Learning (DL)-based… (see more) techniques show promising results by leveraging structural information from the code and learning links between changesets and bug reports. However, since source code associated with changesets evolves, the performance of such models tends to degrade over time due to concept drift. Aiming to address this challenge, in this paper, we evaluate the potential of using Continual Learning (CL) techniques in multiple sub-tasks setting for bug localization (each of which operates on either stationary or non-stationary data), comparing it against a bug localization technique that leverages the BERT model, a deep reinforcement learning-based technique that leverages the A2C algorithm, and a DL-based function-level interaction model for semantic bug localization. Additionally, we enhanced the CL techniques by using logistic regression to identify and integrate the most significant bug-inducing factors. Our empirical evaluation across seven widely used software projects shows that CL techniques perform better than DL-based techniques by up to 61% in terms of Mean Reciprocal Rank (MRR), 44% in terms of Mean Average Precision (MAP), 83% in terms of top@1, 56% in terms of top@5, and 66% in terms of top@10 metrics in non-stationary setting. Further, we show that the CL techniques we studied are effective at localizing changesets relevant to a bug report while being able to mitigate catastrophic forgetting across the studied tasks and require up to 5x less computational effort during training. Our findings demonstrate the potential of adopting CL for bug localization in non-stationary settings, and we hope it helps to improve bug localization activities in Software Engineering using CL techniques.

2025-10-17

ACM Transactions on Software Engineering and Methodology (published)

Refactoring with LLMs: Bridging Human Expertise and Machine Understanding

Yonnel Chen Kuang Piao

Jean Carlors Paul

Leuson Da Silva

Arghavan Moradi Dakhel

Mohammad Hamdaqa

2025-10-03

ArXiv (preprint)

DeepCodeProbe: Evaluating Code Representation Quality in Models Trained on Code

Vahid Majdinasab

Amin Nikanjam

2025-09-29

Empirical Software Engineering (published)

BloomAPR: A Bloom's Taxonomy-based Framework for Assessing the Capabilities of LLM-Powered APR Solutions

Yinghang Ma

Jiho Shin

Leuson Da Silva

Zhen Ming (Jack) Jiang

Song Wang

Shin Hwei Tan

Recent advances in large language models (LLMs) have accelerated the development of AI-driven automated program repair (APR) solutions. Howe… (see more)ver, these solutions are typically evaluated using static benchmarks such as Defects4J and SWE-bench, which suffer from two key limitations: (1) the risk of data contamination, potentially inflating evaluation results due to overlap with LLM training data, and (2) limited ability to assess the APR capabilities in dynamic and diverse contexts. In this paper, we introduced BloomAPR, a novel dynamic evaluation framework grounded in Bloom's Taxonomy. Our framework offers a structured approach to assess the cognitive capabilities of LLM-powered APR solutions across progressively complex reasoning levels. Using Defects4J as a case study, we evaluated two state-of-the-art LLM-powered APR solutions, ChatRepair and CigaR, under three different LLMs: GPT-3.5-Turbo, Llama-3.1, and StarCoder-2. Our findings show that while these solutions exhibit basic reasoning skills and effectively memorize bug-fixing patterns (fixing up to 81.57% of bugs at the Remember layer), their performance increases with synthetically generated bugs (up to 60.66% increase at the Understand layer). However, they perform worse on minor syntactic changes (fixing up to 43.32% at the Apply layer), and they struggle to repair similar bugs when injected into real-world projects (solving only 13.46% to 41.34% bugs at the Analyze layer). These results underscore the urgent need for evolving benchmarks and provide a foundation for more trustworthy evaluation of LLM-powered software engineering solutions.

2025-09-28

ArXiv (preprint)

Towards Understanding the Impact of Data Bugs on Deep Learning Models in Software Engineering

Mehil B. Shah

Mohammad Masudur Rahman

Deep learning (DL) techniques have achieved significant success in various software engineering tasks (e.g., code completion by Copilot). Ho… (see more)wever, DL systems are prone to bugs from many sources, including training data. Existing literature suggests that bugs in training data are highly prevalent, but little research has focused on understanding their impacts on the models used in software engineering tasks. In this paper, we address this research gap through a comprehensive empirical investigation focused on three types of data prevalent in software engineering tasks: code-based, text-based, and metric-based. Using state-of-the-art baselines, we compare the models trained on clean datasets with those trained on datasets with quality issues and without proper preprocessing. By analysing the gradients, weights, and biases from neural networks under training, we identify the symptoms of data quality and preprocessing issues. Our analysis reveals that quality issues in code data cause biased learning and gradient instability, whereas problems in text data lead to overfitting and poor generalisation of models. On the other hand, quality issues in metric data result in exploding gradients and model overfitting, and inadequate preprocessing exacerbates these effects across all three data types. Finally, we demonstrate the validity and generalizability of our findings using six new datasets. Our research provides a better understanding of the impact and symptoms of data bugs in software engineering datasets. Practitioners and researchers can leverage these findings to develop better monitoring systems and data-cleaning methods to help detect and resolve data bugs in deep learning systems.

2025-09-28

Empirical Software Engineering (published)

FairFLRep: Fairness aware fault localization and repair of Deep Neural Networks

Moses Openja

Paolo Arcaini

Fuyuki Ishikawa

2025-09-09

ACM Transactions on Software Engineering and Methodology (published)

Adversarial Attack Classification and Robustness Testing for Large Language Models for Code

Yang Liu

Armstrong Foundjem

Heng Li

Large Language Models (LLMs) have become vital tools in software development tasks such as code generation, completion, and analysis. As the… (see more)ir integration into workflows deepens, ensuring robustness against vulnerabilities especially those triggered by diverse or adversarial inputs becomes increasingly important. Such vulnerabilities may lead to incorrect or insecure code generation when models encounter perturbed task descriptions, code, or comments. Prior research often overlooks the role of natural language in guiding code tasks. This study investigates how adversarial perturbations in natural language inputs including prompts, comments, and descriptions affect LLMs for Code (LLM4Code). It examines the effects of perturbations at the character, word, and sentence levels to identify the most impactful vulnerabilities. We analyzed multiple projects (e.g., ReCode, OpenAttack) and datasets (e.g., HumanEval, MBPP), establishing a taxonomy of adversarial attacks. The first dimension classifies the input type code, prompts, or comments while the second dimension focuses on granularity: character, word, or sentence-level changes. We adopted a mixed-methods approach, combining quantitative performance metrics with qualitative vulnerability analysis. LLM4Code models show varying robustness across perturbation types. Sentence-level attacks were least effective, suggesting models are resilient to broader contextual changes. In contrast, word-level perturbations posed serious challenges, exposing semantic vulnerabilities. Character-level effects varied, showing model sensitivity to subtle syntactic deviations.Our study offers a structured framework for testing LLM4Code robustness and emphasizes the critical role of natural language in adversarial evaluation. Improving model resilience to semantic-level disruptions is essential for secure and reliable code-generation systems.

2025-08-10

Empirical Software Engineering (published)

Deep Reinforcement Learning-Based Intrusion Detection System: Defending Edge Gateways Against Mirai and Gafgyt

Saeid Jamshidi

Amin Nikanjam

Kawser Wazed Nafi

The rapid growth of the Internet of Things (IoT) has transformed industries, resulting in unprecedented opportunities alongside significant … (see more)cybersecurity challenges. Malware, for example, Mirai and Gafgyt, exploits IoT vulnerabilities, leading to large-scale attacks. Traditional Intrusion Detection Systems (IDS) struggle to detect these evolving threats due to their reliance on static rule-based or classic Machine Learning (ML) models, which lack adaptability to zero-day attacks and dynamic traffic patterns. This paper presents EdgeShield-DRL, a novel Deep Reinforcement Learning (DRL)-based IDS designed for IoT edge gateways. EdgeShield-DRL dynamically detects and mitigates evolving threats in real-time while ensuring efficient operation on resource-constrained edge devices. We evaluated EdgeShieldDRL on the N-BaIoT dataset, achieving a high detection accuracy of

2025-08-10

2025 12th International Conference on Future Internet of Things and Cloud (FiCloud) (published)