Foutse Khomh

Joselito Joey Chua

2025-01-01

IEEE Software (publié)

MLOps, LLMOps, FMOps, and Beyond

Chakkrit Tantithamthavorn

Fabio Palomba

Joselito Joey Chua

2025-01-01

IEEE Software (publié)

MLOps, LLMOps, FMOps, and Beyond

Chakkrit Tantithamthavorn

Fabio Palomba

Joselito Joey Chua

2025-01-01

IEEE Software (publié)

Editorial: Special Issue on Software Engineering and AI for Data Quality

Andreas Metzger

Phu H. Nguyen

Sagar Sen

This editorial summarizes the content of the Special Issue on Software Engineering and AI for Data Quality of the Journal of Data and Inform… (voir plus)ation Quality (JDIQ).

2024-12-19

Journal of Data and Information Quality (publié)

Editorial: Special Issue on Software Engineering and AI for Data Quality

Andreas Metzger

Phu Nguyen

Sagar Sen

This editorial summarizes the content of the Special Issue on Software Engineering and AI for Data Quality of the Journal of Data and Inform… (voir plus)ation Quality (JDIQ).

2024-12-19

Journal of Data and Information Quality (publié)

Editorial: Special Issue on Software Engineering and AI for Data Quality

Andreas Metzger

Phu Nguyen

Sagar Sen

This editorial summarizes the content of the Special Issue on Software Engineering and AI for Data Quality of the Journal of Data and Inform… (voir plus)ation Quality (JDIQ).

2024-12-19

Journal of Data and Information Quality (publié)

Leveraging Data Characteristics for Bug Localization in Deep Learning Programs

Ruchira Manke

Mohammad Wardat

Hridesh Rajan

Deep Learning (DL) is a class of machine learning algorithms that are used in a wide variety of applications. Like any software system, DL p… (voir plus)rograms can have bugs. To support bug localization in DL programs, several tools have been proposed in the past. As most of the bugs that occur due to improper model structure known as structural bugs lead to inadequate performance during training, it is challenging for developers to identify the root cause and address these bugs. To support bug detection and localization in DL programs, in this paper, we propose Theia, which detects and localizes structural bugs in DL programs. Unlike the previous works, Theia considers the training dataset characteristics to automatically detect bugs in DL programs developed using two deep learning libraries, Keras and PyTorch . Since training the DL models is a time-consuming process, Theia detects these bugs at the beginning of the training process and alerts the developer with informative messages containing the bug's location and actionable fixes which will help them to improve the structure of the model. We evaluated Theia on a benchmark of 40 real-world buggy DL programs obtained from Stack Overflow . Our results show that Theia successfully localizes 57/75 structural bugs in 40 buggy programs, whereas NeuraLint, a state-of-the-art approach capable of localizing structural bugs before training localizes 17/75 bugs.

2024-12-16

ACM Transactions on Software Engineering and Methodology (publié)

Continuously Learning Bug Locations

Paulina Stevia Nouwou Mindom

Léuson M. P. Da Silva

Amin Nikanjam

Automatically locating buggy changesets associated with bug reports is crucial in the software development process. Deep Learning (DL)-based… (voir plus) techniques show promising results by leveraging structural information from the code and learning links between changesets and bug reports. However, since source code associated with changesets evolves, the performance of such models tends to degrade over time due to concept drift. Aiming to address this challenge, in this paper, we evaluate the potential of using Continual Learning (CL) techniques in multiple sub-tasks setting for bug localization (each of which operates on either stationary or non-stationary data), comparing it against a bug localization technique that leverages the BERT model, a deep reinforcement learning-based technique that leverages the A2C algorithm, and a DL-based function-level interaction model for semantic bug localization. Additionally, we enhanced the CL techniques by using logistic regression to identify and integrate the most significant bug-inducing factors. Our empirical evaluation across seven widely used software projects shows that CL techniques perform better than DL-based techniques by up to 61% in terms of Mean Reciprocal Rank (MRR), 44% in terms of Mean Average Precision (MAP), 83% in terms of top@1, 56% in terms of top@5, and 66% in terms of top@10 metrics in non-stationary setting. Further, we show that the CL techniques we studied are effective at localizing changesets relevant to a bug report while being able to mitigate catastrophic forgetting across the studied tasks and require up to 5x less computational effort during training. Our findings demonstrate the potential of adopting CL for bug localization in non-stationary settings, and we hope it helps to improve bug localization activities in Software Engineering using CL techniques.

2024-12-15

ArXiv (prépublication)

Continuously Learning Bug Locations

Paulina Stevia Nouwou Mindom

Léuson M. P. Da Silva

Amin Nikanjam

Automatically locating buggy changesets associated with bug reports is crucial in the software development process. Deep Learning (DL)-based… (voir plus) techniques show promising results by leveraging structural information from the code and learning links between changesets and bug reports. However, since source code associated with changesets evolves, the performance of such models tends to degrade over time due to concept drift. Aiming to address this challenge, in this paper, we evaluate the potential of using Continual Learning (CL) techniques in multiple sub-tasks setting for bug localization (each of which operates on either stationary or non-stationary data), comparing it against a bug localization technique that leverages the BERT model, a deep reinforcement learning-based technique that leverages the A2C algorithm, and a DL-based function-level interaction model for semantic bug localization. Additionally, we enhanced the CL techniques by using logistic regression to identify and integrate the most significant bug-inducing factors. Our empirical evaluation across seven widely used software projects shows that CL techniques perform better than DL-based techniques by up to 61% in terms of Mean Reciprocal Rank (MRR), 44% in terms of Mean Average Precision (MAP), 83% in terms of top@1, 56% in terms of top@5, and 66% in terms of top@10 metrics in non-stationary setting. Further, we show that the CL techniques we studied are effective at localizing changesets relevant to a bug report while being able to mitigate catastrophic forgetting across the studied tasks and require up to 5x less computational effort during training. Our findings demonstrate the potential of adopting CL for bug localization in non-stationary settings, and we hope it helps to improve bug localization activities in Software Engineering using CL techniques.

2024-12-15

ArXiv (prépublication)

Continuously Learning Bug Locations

Paulina Stevia Nouwou Mindom

Leuson Da Silva

Amin Nikanjam

Automatically locating buggy changesets associated with bug reports is crucial in the software development process. Deep Learning (DL)-based… (voir plus) techniques show promising results by leveraging structural information from the code and learning links between changesets and bug reports. However, since source code associated with changesets evolves, the performance of such models tends to degrade over time due to concept drift. Aiming to address this challenge, in this paper, we evaluate the potential of using Continual Learning (CL) techniques in multiple sub-tasks setting for bug localization (each of which operates on either stationary or non-stationary data), comparing it against a bug localization technique that leverages the BERT model, a deep reinforcement learning-based technique that leverages the A2C algorithm, and a DL-based function-level interaction model for semantic bug localization. Additionally, we enhanced the CL techniques by using logistic regression to identify and integrate the most significant bug-inducing factors. Our empirical evaluation across seven widely used software projects shows that CL techniques perform better than DL-based techniques by up to 61% in terms of Mean Reciprocal Rank (MRR), 44% in terms of Mean Average Precision (MAP), 83% in terms of top@1, 56% in terms of top@5, and 66% in terms of top@10 metrics in non-stationary setting. Further, we show that the CL techniques we studied are effective at localizing changesets relevant to a bug report while being able to mitigate catastrophic forgetting across the studied tasks and require up to 5x less computational effort during training. Our findings demonstrate the potential of adopting CL for bug localization in non-stationary settings, and we hope it helps to improve bug localization activities in Software Engineering using CL techniques.

2024-12-15

ArXiv (prépublication)

Harnessing pre-trained generalist agents for software engineering tasks

Paulina Stevia Nouwou Mindom

Amin Nikanjam

2024-12-11

Empirical Software Engineering (publié)

Harnessing Pre-trained Generalist Agents for Software Engineering Tasks

Paulina Stevia Nouwou Mindom

Amin Nikanjam

Nowadays, we are witnessing an increasing adoption of Artificial Intelligence (AI) to develop techniques aimed at improving the reliability,… (voir plus) effectiveness, and overall quality of software systems. Deep reinforcement learning (DRL) has recently been successfully used for automation in complex tasks such as game testing and solving the job-shop scheduling problem. However, these specialized DRL agents, trained from scratch on specific tasks, suffer from a lack of generalizability to other tasks and they need substantial time to be developed and re-trained effectively. Recently, DRL researchers have begun to develop generalist agents, able to learn a policy from various environments and capable of achieving performances similar to or better than specialist agents in new tasks. In the Natural Language Processing or Computer Vision domain, these generalist agents are showing promising adaptation capabilities to never-before-seen tasks after a light fine-tuning phase and achieving high performance. This paper investigates the potential of generalist agents for solving SE tasks. Specifically, we conduct an empirical study aimed at assessing the performance of two generalist agents on two important SE tasks: the detection of bugs in games (for two games) and the minimization of makespan in a scheduling task, to solve the job-shop scheduling problem (for two instances). Our results show that the generalist agents outperform the specialist agents with very little effort for fine-tuning, achieving a 20% reduction of the makespan over specialized agent performance on task-based scheduling. In the context of game testing, some generalist agent configurations detect 85% more bugs than the specialist agents. Building on our analysis, we provide recommendations for researchers and practitioners looking to select generalist agents for SE tasks, to ensure that they perform effectively.

2024-12-11

Empirical Software Engineering (publié)