Foutse Khomh

Biographie

Foutse Khomh est professeur titulaire de génie logiciel à Polytechnique Montréal, titulaire d'une chaire en IA Canada-CIFAR dans le domaine des systèmes logiciels d'apprentissage automatique fiables, et titulaire d'une chaire de recherche FRQ-IVADO sur l'assurance qualité des logiciels pour les applications d'apprentissage automatique.

Il a obtenu un doctorat en génie logiciel de l'Université de Montréal en 2011, avec une bourse d'excellence. Il a également reçu le prix CS-Can/Info-Can du meilleur jeune chercheur en informatique en 2019. Ses recherches portent sur la maintenance et l'évolution des logiciels, l'ingénierie des systèmes d'apprentissage automatique, l'ingénierie en nuage et l’IA/apprentissage automatique fiable et digne de confiance.

Ses travaux ont été récompensés par quatre prix de l’article le plus important Most Influential Paper en dix ans et six prix du meilleur article ou de l’article exceptionnel (Best/Distinguished Paper). Il a également siégé au comité directeur de plusieurs conférences et rencontres : SANER (comme président), MSR, PROMISE, ICPC (comme président) et ICSME (en tant que vice-président). Il a initié et coorganisé le symposium Software Engineering for Machine Learning Applications (SEMLA) et la série d'ateliers Release Engineering (RELENG).

Il est cofondateur du projet CRSNG CREATE SE4AI : A Training Program on the Development, Deployment, and Servicing of Artificial Intelligence-based Software Systems et l'un des chercheurs principaux du projet Dependable Explainable Learning (DEEL). Il est également cofondateur de l'initiative québécoise sur l'IA digne de confiance (Confiance IA Québec). Il fait partie du comité de rédaction de plusieurs revues internationales de génie logiciel (dont IEEE Software, EMSE, JSEP) et est membre senior de l'Institute of Electrical and Electronics Engineers (IEEE).

Étudiants actuels

Nanda Assobjio Brice Yvan

Collaborateur·rice alumni - Polytechnique

Gabriel Laberge

Doctorat - Polytechnique

Github

forough majidi

Doctorat - Polytechnique

Site web

mohammadhossein.malekpour@gmail.com

Mo Malekpour

Maîtrise recherche - Polytechnique

Site web

Github

Elnathan Tiokou Tiokou Fangang

Mohamed Amine Merzouk

Postdoctorat - Polytechnique

Co-superviseur⋅e :

Maîtrise recherche - Polytechnique

Doctorat - Polytechnique

Github

Ben Braiek Yasmine

Maîtrise recherche - Polytechnique

Publications

Mock Deep Testing: Toward Separate Development of Data and Models for Deep Learning

Ruchira Manke

Mohammad Wardat

Hridesh Rajan

While deep learning (DL) has permeated, and become an integral component of many critical software systems, today software engineering resea… (voir plus)rch hasn't explored how to separately test data and models that are integral for DL approaches to work effectively. The main challenge in independently testing these components arises from the tight dependency between data and models. This research explores this gap, introducing our methodology of mock deep testing for unit testing of DL applications. To enable unit testing, we introduce a design paradigm that decomposes the workflow into distinct, manageable components, minimizes sequential dependencies, and modularizes key stages of the DL. For unit testing these components, we propose modeling their dependencies using mocks. This modular approach facilitates independent development and testing of the components, ensuring comprehensive quality assurance throughout the development process. We have developed KUnit, a framework for enabling mock deep testing for the Keras library. We empirically evaluated KUnit to determine the effectiveness of mocks. Our assessment of 50 DL programs obtained from Stack Overflow and GitHub shows that mocks effectively identified 10 issues in the data preparation stage and 53 issues in the model design stage. We also conducted a user study with 36 participants using KUnit to perceive the effectiveness of our approach. Participants using KUnit successfully resolved 25 issues in the data preparation stage and 38 issues in the model design stage. Our findings highlight that mock objects provide a lightweight emulation of the dependencies for unit testing, facilitating early bug detection. Lastly, to evaluate the usability of KUnit, we conducted a post-study survey. The results reveal that KUnit is helpful to DL application developers, enabling them to independently test each component effectively in different stages.

2025-02-11

ArXiv (prépublication)

Application of deep reinforcement learning for intrusion detection in Internet of Things: A systematic review

Saeid Jamshidi

Amin Nikanjam

Kawser Wazed Nafi

Rasoul Rasta

2025-02-01

Internet of Things (publié)

An empirical study of testing machine learning in the wild

Moses Openja

Armstrong Foundjem

Zhen Ming (Jack) Jiang

Mouna Abidi

Ahmed E. Hassan

Background: Recently, machine and deep learning (ML/DL) algorithms have been increasingly adopted in many software systems. Due to their in… (voir plus)ductive nature, ensuring the quality of these systems remains a significant challenge for the research community. Traditionally, software systems were constructed deductively, by writing explicit rules that govern the behavior of the system as program code. However, ML/DL systems infer rules from training data i.e., they are generated inductively). Recent research in ML/DL quality assurance has adapted concepts from traditional software testing, such as mutation testing, to improve reliability. However, it is unclear if these proposed testing techniques are adopted in practice, or if new testing strategies have emerged from real-world ML deployments. There is little empirical evidence about the testing strategies. Aims: To fill this gap, we perform the first fine-grained empirical study on ML testing in the wild to identify the ML properties being tested, the testing strategies, and their implementation throughout the ML workflow. Method: We conducted a mixed-methods study to understand ML software testing practices. We analyzed test files and cases from 11 open-source ML/DL projects on GitHub. Using open coding, we manually examined the testing strategies, tested ML properties, and implemented testing methods to understand their practical application in building and releasing ML/DL software systems. Results: Our findings reveal several key insights: 1.) The most common testing strategies, accounting for less than 40%, are Grey-box and White-box methods, such as Negative Testing , Oracle Approximation , and Statistical Testing . 2.) A wide range of

$17$ ML properties are tested, out of which only 20% to 30% are frequently tested, including Consistency , Correctness , and Efficiency . 3.) Bias and Fairness is more tested in Recommendation (6%) and CV (3.9%) systems, while Security & Privacy is tested in CV (2%), Application Platforms (0.9%), and NLP (0.5%). 4.) We identified 13 types of testing methods, such as Unit Testing , Input Testing , and Model Testing . Conclusions: This study sheds light on the current adoption of software testing techniques and highlights gaps and limitations in existing ML testing practices.

2025-01-01

ACM Trans. Softw. Eng. Methodol. (publié)

Evaluating machine learning-driven intrusion detection systems in IoT: Performance and energy consumption

Saeid Jamshidi

Kawser Wazed Nafi

Amin Nikanjam

2025-01-01

Comput. Ind. Eng. (publié)

MLOps, LLMOps, FMOps, and Beyond

Chakkrit Tantithamthavorn

Fabio Palomba

Joselito Joey Chua

2025-01-01

IEEE Software (publié)

MLOps, LLMOps, FMOps, and Beyond

Chakkrit Kla Tantithamthavorn

Fabio Palomba

Joselito Joey Chua

2025-01-01

IEEE Software (publié)

MLOps, LLMOps, FMOps, and Beyond

Chakkrit Tantithamthavorn

Fabio Palomba

Joselito Joey Chua

2025-01-01

IEEE Software (publié)

Editorial: Special Issue on Software Engineering and AI for Data Quality

Andreas Metzger

Phu Nguyen

Sagar Sen

This editorial summarizes the content of the Special Issue on Software Engineering and AI for Data Quality of the Journal of Data and Inform… (voir plus)ation Quality (JDIQ).

2024-12-19

Journal of Data and Information Quality (publié)

Editorial: Special Issue on Software Engineering and AI for Data Quality

Andreas Metzger

Phu H. Nguyen

Sagar Sen

This editorial summarizes the content of the Special Issue on Software Engineering and AI for Data Quality of the Journal of Data and Inform… (voir plus)ation Quality (JDIQ).

2024-12-19

Journal of Data and Information Quality (publié)

Editorial: Special Issue on Software Engineering and AI for Data Quality

Andreas Metzger

Phu Nguyen

Sagar Sen

This editorial summarizes the content of the Special Issue on Software Engineering and AI for Data Quality of the Journal of Data and Inform… (voir plus)ation Quality (JDIQ).

2024-12-19

Journal of Data and Information Quality (publié)

Leveraging Data Characteristics for Bug Localization in Deep Learning Programs

Ruchira Manke

Mohammad Wardat

Hridesh Rajan

Deep Learning (DL) is a class of machine learning algorithms that are used in a wide variety of applications. Like any software system, DL p… (voir plus)rograms can have bugs. To support bug localization in DL programs, several tools have been proposed in the past. As most of the bugs that occur due to improper model structure known as structural bugs lead to inadequate performance during training, it is challenging for developers to identify the root cause and address these bugs. To support bug detection and localization in DL programs, in this paper, we propose Theia, which detects and localizes structural bugs in DL programs. Unlike the previous works, Theia considers the training dataset characteristics to automatically detect bugs in DL programs developed using two deep learning libraries, Keras and PyTorch . Since training the DL models is a time-consuming process, Theia detects these bugs at the beginning of the training process and alerts the developer with informative messages containing the bug's location and actionable fixes which will help them to improve the structure of the model. We evaluated Theia on a benchmark of 40 real-world buggy DL programs obtained from Stack Overflow . Our results show that Theia successfully localizes 57/75 structural bugs in 40 buggy programs, whereas NeuraLint, a state-of-the-art approach capable of localizing structural bugs before training localizes 17/75 bugs.

2024-12-16

ACM Transactions on Software Engineering and Methodology (publié)

Continuously Learning Bug Locations

Paulina Stevia Nouwou Mindom

Leuson Da Silva

Amin Nikanjam

Automatically locating buggy changesets associated with bug reports is crucial in the software development process. Deep Learning (DL)-based… (voir plus) techniques show promising results by leveraging structural information from the code and learning links between changesets and bug reports. However, since source code associated with changesets evolves, the performance of such models tends to degrade over time due to concept drift. Aiming to address this challenge, in this paper, we evaluate the potential of using Continual Learning (CL) techniques in multiple sub-tasks setting for bug localization (each of which operates on either stationary or non-stationary data), comparing it against a bug localization technique that leverages the BERT model, a deep reinforcement learning-based technique that leverages the A2C algorithm, and a DL-based function-level interaction model for semantic bug localization. Additionally, we enhanced the CL techniques by using logistic regression to identify and integrate the most significant bug-inducing factors. Our empirical evaluation across seven widely used software projects shows that CL techniques perform better than DL-based techniques by up to 61% in terms of Mean Reciprocal Rank (MRR), 44% in terms of Mean Average Precision (MAP), 83% in terms of top@1, 56% in terms of top@5, and 66% in terms of top@10 metrics in non-stationary setting. Further, we show that the CL techniques we studied are effective at localizing changesets relevant to a bug report while being able to mitigate catastrophic forgetting across the studied tasks and require up to 5x less computational effort during training. Our findings demonstrate the potential of adopting CL for bug localization in non-stationary settings, and we hope it helps to improve bug localization activities in Software Engineering using CL techniques.

2024-12-15

ArXiv (prépublication)