Merits of organizational metrics in defect prediction: an industrial replication

Bora Caglayan, Burak Turhan, Ayse Bener, Mayy Habayeb, Andriy Miransky, Enzo Cialini

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

23 Citations (Scopus)


Defect prediction models presented in the literature lack generalization unless the original study can be replicated using new datasets and in different organizational settings. Practitioners can also benefit from replicating studies in their own environment by gaining insights and comparing their findings with those reported. In this work, we replicated an earlier study in order to investigate the merits of organizational metrics in building defect prediction models for large-scale enterprise software. We mined the organizational, code complexity, code churn and pre-release bug metrics of that large scale software and built defect prediction models for each metric set. In the original study, organizational metrics were found to achieve the highest performance. In our case, models based on organizational metrics performed better than models based on churn metrics but were outperformed by pre-release metric models. Further, we verified four individual organizational metrics as indicators for defects. We conclude that the performance of different metric sets in building defect prediction models depends on the project's characteristics and the targeted prediction level. Our replication of earlier research enabled assessing the validity and limitations of organizational metrics in a different context.

Original languageEnglish
Title of host publicationProceedings
Subtitle of host publication2015 IEEE/ACM 37th IEEE International Conference on Software Engineering - ICSE 2015
EditorsSebastian Elbaum, Gerardo Canfora
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages10
ISBN (Electronic)9781479919345
Publication statusPublished - 2015
Externally publishedYes
EventInternational Conference on Software Engineering 2015 - Florence, Italy
Duration: 16 May 201524 May 2015
Conference number: 37th (Proceedings)


ConferenceInternational Conference on Software Engineering 2015
Abbreviated titleICSE 2015
Internet address


  • Defect prediction
  • Model comparison
  • Organizational metrics
  • Replication
  • Software engineering

Cite this