Practitioners' perceptions of the goals and visual explanations of defect prediction models

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

58 Citations (Scopus)

Abstract

Software defect prediction models are classifiers that are constructed from historical software data. Such software defect prediction models have been proposed to help developers optimize the limited Software Quality Assurance (SQA) resources and help managers develop SQA plans. Prior studies have different goals for their defect prediction models and use different techniques for generating visual explanations of their models. Yet, it is unclear what are the practitioners' perceptions of (1) these defect prediction model goals, and (2) the model-agnostic techniques used to visualize these models. We conducted a qualitative survey to investigate practitioners' perceptions of the goals of defect prediction models and the model-agnostic techniques used to generate visual explanations of defect prediction models. We found that (1) 82%-84% of the respondents perceived that the three goals of defect prediction models are useful; (2) LIME is the most preferred technique for understanding the most important characteristics that contributed to a prediction of a file, while ANOVA/VarImp is the second most preferred technique for understanding the characteristics that are associated with software defects in the past. Our findings highlight the significance of investigating how to improve the understanding of defect prediction models and their predictions. Hence, model-agnostic techniques from explainable AI domain may help practitioners to understand defect prediction models and their predictions.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021
EditorsKelly Blincoe, Meiyappan Nagappan
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages432-443
Number of pages12
ISBN (Electronic)9781728187105
ISBN (Print)9781665429856
DOIs
Publication statusPublished - 2021
EventIEEE International Working Conference on Mining Software Repositories 2021 - Online, Madrid, Spain
Duration: 22 May 202130 May 2021
Conference number: 18th
https://ieeexplore-ieee-org.ezproxy.lib.monash.edu.au/xpl/conhome/9463061/proceeding (Proceedings)

Publication series

NameProceedings - 2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)2574-3848
ISSN (Electronic)2574-3864

Conference

ConferenceIEEE International Working Conference on Mining Software Repositories 2021
Abbreviated titleMSR 2021
Country/TerritorySpain
CityMadrid
Period22/05/2130/05/21
Internet address

Keywords

  • Defect Prediction
  • Explainable AI
  • Software Analytics
  • Software Quality Assurance

Cite this