An exploratory study of search based training data selection for cross project defect prediction

Seyedrebvar Hosseini, Burak Turhan

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Context: Search based approaches are gaining attention in cross project defect prediction (CPDP). The complexity of such approaches and existence of various design decisions are important issues to consider. Objective: We aim at investigating factors that can affect the performance of search based selection (SBS) approaches. We study a genetic instance selection approach (GIS) and present an evaluation of design options for search based CPDP. Method: Using an exploratory approach, data from different options of models are gathered and analyzed through ANOVA tests and effect sizes. Results: Both feature sets and validation dataset selection options show small or insignificant impacts on F-measure and precision, unlike the more affected false positive and true negative rates. Size of training data does not seem to be related to significant changes in F-measure and precision and high variability in performance are discouraging evidence for using larger datasets. Fitness function is one of the major factors that impact performance with much larger effect than the choice of validation dataset. Finally, while showing slight impacts, data label changes do not seem to be the top contributor to performance. Conclusions: We conclude that exploratory approaches can be effective for making design decisions in constructing search based CPDP models. Effect of individual tuned learners and their interaction with other affecting parameters and more in depth study of quality affecting factors guided by label changes are directions to investigate.

Original languageEnglish
Title of host publicationProceedings - 44th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2018
Subtitle of host publication29–31 August 2018 Prague, Czech Republic
EditorsTomas Bures, Lefteris Angelis
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages244-251
Number of pages8
ISBN (Electronic)9781538673829
ISBN (Print)9781538673843
DOIs
Publication statusPublished - 2018
Externally publishedYes
EventEuromicro Conference on Software Engineering and Advanced Applications 2018 - Prague, Czech Republic
Duration: 29 Aug 201831 Aug 2018
Conference number: 44th
http://dsd-seaa2018.fit.cvut.cz/seaa/

Conference

ConferenceEuromicro Conference on Software Engineering and Advanced Applications 2018
Abbreviated titleSEAA 2018
CountryCzech Republic
CityPrague
Period29/08/1831/08/18
Internet address

Keywords

  • Cross project defect prediction
  • Exploratory search based optimization
  • Training data selection

Cite this

Hosseini, S., & Turhan, B. (2018). An exploratory study of search based training data selection for cross project defect prediction. In T. Bures, & L. Angelis (Eds.), Proceedings - 44th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2018: 29–31 August 2018 Prague, Czech Republic (pp. 244-251). [8498215] IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/SEAA.2018.00048