Foutse Khomh

Biography

Foutse Khomh is a full professor of software engineering at Polytechnique Montréal, a Canada CIFAR AI Chair – Trustworthy Machine Learning Software Systems, and an FRQ-IVADO Research Chair in Software Quality Assurance for Machine Learning Applications. Khomh completed a PhD in software engineering at Université de Montréal in 2011, for which he received an Award of Excellence. He was also awarded a CS-Can/Info-Can Outstanding Young Computer Science Researcher Prize in 2019.

His research interests include software maintenance and evolution, machine learning systems engineering, cloud engineering, and dependable and trustworthy ML/AI. His work has received four Ten-year Most Influential Paper (MIP) awards, and six Best/Distinguished Paper Awards. He has served on the steering committee of numerous organizations in software engineering, including SANER (chair), MSR, PROMISE, ICPC (chair), and ICSME (vice-chair). He initiated and co-organized Polytechnique Montréal‘s Software Engineering for Machine Learning Applications (SEMLA) symposium and the RELENG (release engineering) workshop series.

Khomh co-founded the NSERC CREATE SE4AI: A Training Program on the Development, Deployment and Servicing of Artificial Intelligence-based Software Systems, and is a principal investigator for the DEpendable Explainable Learning (DEEL) project.

He also co-founded Confiance IA, a Quebec consortium focused on building trustworthy AI, and is on the editorial board of multiple international software engineering journals, including IEEE Software, EMSE and JSEP. He is a senior member of IEEE.

Current Students

Houssem Ben Braiek

Postdoctorate - Polytechnique Montréal

PhD - Polytechnique Montréal

Github

forough majidi

PhD - Polytechnique Montréal

Website

Khouloud Oueslati

Master's Research - Polytechnique Montréal

Arian Qazvini

Master's Research - Polytechnique Montréal

Website

Github

Elnathan Tiokou Tiokou Fangang

Master's Research - Polytechnique Montréal

Ben Braiek Yasmine

Master's Research - Polytechnique Montréal

Publications

Physics-Guided Adversarial Machine Learning for Aircraft Systems Simulation

Houssem Ben Braiek

Thomas Reid

In the context of aircraft system performance assessment, deep learning technologies allow us to quickly infer models from experimental meas… (see more)urements, with less detailed system knowledge than usually required by physics-based modeling. However, this inexpensive model development also comes with new challenges regarding model trustworthiness. This article presents a novel approach, physics-guided adversarial machine learning (ML), which improves the confidence over the physics consistency of the model. The approach performs, first, a physics-guided adversarial testing phase to search for test inputs revealing behavioral system inconsistencies, while still falling within the range of foreseeable operational conditions. Then, it proceeds with a physics-informed adversarial training to teach the model the system-related physics domain foreknowledge through iteratively reducing the unwanted output deviations on the previously uncovered counterexamples. Empirical evaluation on two aircraft system performance models shows the effectiveness of our adversarial ML approach in exposing physical inconsistencies of both the models and in improving their propensity to be consistent with physics domain knowledge.

2022-12-31

IEEE Transactions on Reliability (published)

PaReco: patched clones and missed patches among the divergent variants of a software family

Poedjadevie Kadjel Ramkisoen

John Businge

Brent van Bladel

Alexandre Decan

Serge Demeyer

Coen De Roover

Re-using whole repositories as a starting point for new projects is often done by maintaining a variant fork parallel to the original. Howev… (see more)er, the common artifacts between both are not always kept up to date. As a result, patches are not optimally integrated across the two repositories, which may lead to sub-optimal maintenance between the variant and the original project. A bug existing in both repositories can be patched in one but not the other (we see this as a missed opportunity) or it can be manually patched in both probably by different developers (we see this as effort duplication). In this paper we present a tool (named PaReCo) which relies on clone detection to mine cases of missed opportunity and effort duplication from a pool of patches. We analyzed 364 (source to target) variant pairs with 8,323 patches resulting in a curated dataset containing 1,116 cases of effort duplication and 1,008 cases of missed opportunities. We achieve a precision of 91%, recall of 80%, accuracy of 88%, and F1-score of 85%. Furthermore, we investigated the time interval between patches and found out that, on average, missed patches in the target variants have been introduced in the source variants 52 weeks earlier. Consequently, PaReCo can be used to manage variability in “time” by automatically identifying interesting patches in later project releases to be backported to supported earlier releases.

2022-11-06

ESEC/SIGSOFT FSE (published)

Revisiting the Impact of Anti-patterns on Fault-Proneness: A Differentiated Replication

Aurel Ikama

Vincent Du

Philippe Belias

Biruk Asmare Muse

Mohammad Hamdaqa

Anti-patterns manifesting on software code through code smells have been investigated in terms of their prevalence, detection, refactoring, … (see more)and impact on software quality attributes. In particular, leveraging heuristics to identify fault-fixing commits, Khomh et al. have found that anti-patterns and code smells have an impact on the fault-proneness of a software system. Similarly, Saboury et al. found a relationship between anti-pattern occurrences and fault-proneness, using heuristic to identify fault-fixing commits and fault-inducing changes. However, recent studies question the accuracy of heuristics, and thus the validity of empirical studies that leverage it. Hence, in this work, we would like to investigate to what extent the results of empirical studies using heuristics to identify bug fix commits are affected by the limitations of the heuristics based approach using manually validated bug fix commits as a ground truth. In particular, we conduct a differentiated replication of the work by Khomh et al. We particularly focused on the impact of anti-patterns on fault-proneness as it is the only dependent variable that may be affected by noise in the collected faults data. In our differentiated replication study, (1) we expanded the number of subject systems from 5 to 38, (2) utilized a manually validated dataset of bug-fixing commits from the work of Herbold et al., and (3) answered research questions from Khomh et al., that are related to the relationship between anti-pattern occurrences and fault-proneness. (4) We added an additional research question to investigate if combining results from several heuristic-based approaches could help reduce the impact of noise. Our findings show that the impact of the noise generated by the automatic algorithm heuristic based is negligible for the studied subject systems; meaning that the reported relation observed on noisy data still holds on the clean data. However, we also observed that combining results from several heuristic based approaches do not reduce this noise, quite the contrary.

2022-10-02

2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM) (published)

FIXME: synchronize with database! An empirical study of data access self-admitted technical debt

Biruk Asmare Muse

Csaba Nagy

Anthony Cleve

Giuliano Antoniol

2022-07-07

Empirical Software Engineering (published)

Studying the Practices of Deploying Machine Learning Projects on Docker

Moses Openja

forough majidi

Bhagya Chembakottu

Heng Li

Docker is a containerization service that allows for convenient deployment of websites, databases, applications' APIs, and machine learning … (see more)(ML) models with a few lines of code. Studies have recently explored the use of Docker for deploying general software projects with no specific focus on how Docker is used to deploy ML-based projects. In this study, we conducted an exploratory study to understand how Docker is being used to deploy ML-based projects. As the initial step, we examined the categories of ML-based projects that use Docker. We then examined why and how these projects use Docker, and the characteristics of the resulting Docker images. Our results indicate that six categories of ML-based projects use Docker for deployment, including ML Applications, MLOps/ AIOps, Toolkits, DL Frameworks, Models, and Documentation. We derived the taxonomy of 21 major categories representing the purposes of using Docker, including those specific to models such as model management tasks (e.g., testing, training). We then showed that ML engineers use Docker images mostly to help with the platform portability, such as transferring the software across the operating systems, runtimes such as GPU, and language constraints. However, we also found that more resources may be required to run the Docker images for building ML-based software projects due to the large number of files contained in the image layers with deeply nested directories. We hope to shed light on the emerging practices of deploying ML software projects using containers and highlight aspects that should be improved.

2022-06-12

The International Conference on Evaluation and Assessment in Software Engineering 2022 (published)

Works for Me! Cannot Reproduce – A Large Scale Empirical Study of Non-reproducible Bugs

Mohammad Masudur Rahman

Marco Castelluccio

2022-05-29

Empirical Software Engineering (published)

Challenges in machine learning application development

Md Saidur Rahman

Emilio Rivera

Yann-Gaël Guéhéneuc

Bernd Lehnert

SAP is the market leader in enterprise application software offering an end-to-end suite of applications and services to enable their custom… (see more)ers worldwide to operate their business. Especially, retail customers of SAP deal with millions of sales transactions for their day-to-day business. Transactions are created during retail sales at the point of sale (POS) terminals and those transactions are then sent to some central servers for validations and other business operations. A considerable proportion of the retail transactions may have inconsistencies or anomalies due to many technical and human errors. SAP provides an automated process for error detection but still requires a manual process by dedicated employees using workbench software for correction. However, manual corrections of these errors are time-consuming, labor-intensive, and might be prone to further errors due to incorrect modifications. Thus, automated detection and correction of transaction errors are very important regarding their potential business values and the improvement in the business workflow. In this paper, we report on our experience from a project where we develop an AI-based system to automatically detect transaction errors and propose corrections. We identify and discuss the challenges that we faced during this collaborative research and development project, from two distinct perspectives: Software Engineering and Machine Learning. We report on our experience and insights from the project with guidelines for the identified challenges. We collect developers’ feedback for qualitative analysis of our findings. We believe that our findings and recommendations can help other researchers and practitioners embarking into similar endeavours. CCS CONCEPTS • Software and its engineering → Programming teams.

2022-04-30

2022 IEEE/ACM 1st International Workshop on Software Engineering for Responsible Artificial Intelligence (SE4RAI) (published)

Identification of out-of-distribution cases of CNN using class-based surprise adequacy

Mira Marhaba

Ettore Merlo

Giuliano Antoniol

Machine learning is vulnerable to possible incorrect classification of cases that are out of the distribution observed during training and c… (see more)alibration

2022-04-30

2022 IEEE/ACM 1st International Conference on AI Engineering – Software Engineering for AI (CAIN) (published)

Clones in Deep Learning Code: What, Where, and Why?

Hadhemi Jebnoun

Md Saidur Rahman

Biruk Asmare Muse

Deep Learning applications are becoming increasingly popular. Developers of deep learning systems strive to write more efficient code. Deep … (see more)learning systems are constantly evolving, imposing tighter development timelines and increasing complexity, which may lead to bad design decisions. A copy-paste approach is widely used among deep learning developers because they rely on common frameworks and duplicate similar tasks. Developers often fail to properly propagate changes to all clones fragments during a maintenance activity. To our knowledge, no study has examined code cloning practices in deep learning development. Given the negative impacts of clones on software quality reported in the studies on traditional systems, it is very important to understand the characteristics and potential impacts of code clones on deep learning systems. To this end, we use the NiCad tool to detect clones from 59 Python, 14 C# and 6 Java-based deep learning systems and an equal number of traditional software systems. We then analyze the frequency and distribution of code clones in deep learning and traditional systems. We do further analysis of the distribution of code clones using location-based taxonomy. We also study the correlation between bugs and code clones to assess the impacts of clones on the quality of the studied systems. Finally, we introduce a code clone taxonomy related to deep learning programs and identify the deep learning system development phases in which cloning has the highest risk of faults. Our results show that code cloning is a frequent practice in deep learning systems and that deep learning developers often clone code from files in distant repositories in the system. In addition, we found that code cloning occurs more frequently during DL model construction. And that hyperparameters setting is the phase during which cloning is the riskiest, since it often leads to faults.

2022-04-07

Empirical Software Engineering (published)

Software-Engineering Design Patterns for Machine Learning Applications

Hironori Washizaki

Yann‐Gaël Guéhéneuc

Hironori Takeuchi

Naotake Natori

Takuo Doi

Satoshi Okuda

In this study, a multivocal literature review identified 15 software-engineering design patterns for machine learning applications. Findings… (see more) suggest that there are opportunities to increase the patterns’ adoption in practice by raising awareness of such patterns within the community.

2022-02-28

Computer (published)

On the Performance Implications of Deploying IoT Apps as FaaS

Mohab Aly

Soumaya Yacout

2022-02-17

Communications in Computer and Information Science (published)

Faults in deep reinforcement learning programs: a taxonomy and a detection approach

Amin Nikanjam

Mohammad Mehdi Morovati

Houssem Ben Braiek

2021-12-19

Automated Software Engineering (published)