A replicated experiment on the effectiveness of test-first development

Davide Fucci, Burak Turhan

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

19 Citations (Scopus)


Background: Test-first development (TF) is regarded as a development practice that can lead to better quality of software products, as well as improved developer productivity. By implementing unit tests before the corresponding production code, the tests themselves are the main driver to such improvements. The role of tests on the effectiveness of TF has been studied in a controlled experiment by Erdogmus et al. (i.e. original study). Aim: Our goal is to examine the impact of test-first (TF) development on product quality and developer productivity, specifically the role that tests play in it. Method: We replicated the original study's controlled experiment by comparing an experimental group applying TF to a control group applying a test-last approach. We then carried out a correlation study in order to understand whether the number of tests is a good predictor for external quality and/or productivity. Results: Mann-Whitney tests did not show any significant difference between the two groups in terms of number of tests written (W=114.5, p=0.38), developers' productivity (W=90, p=0.82) and external quality (W=81.55, p=0.53). In addition, while a significant correlation exists between the number of tests and productivity (Spearman's ? = 0.57, p

Original languageEnglish
Title of host publication2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages10
ISBN (Print)9781479911448
Publication statusPublished - 1 Dec 2013
Externally publishedYes
EventInternational Symposium on Empirical Software Engineering and Measurement 2013 - Baltimore, United States of America
Duration: 10 Oct 201311 Oct 2013


ConferenceInternational Symposium on Empirical Software Engineering and Measurement 2013
Abbreviated titleESEM 2013
CountryUnited States of America


  • experiment replication
  • productivity
  • software quality
  • test-driven development

Cite this