A replicated experiment on the effectiveness of test-first development

Davide Fucci, Burak Turhan

Research output: Contribution to journalConference articleResearchpeer-review

Abstract

Background: Test-first development (TF) is regarded as a development practice that can lead to better quality of software products, as well as improved developer productivity. By implementing unit tests before the corresponding production code, the tests themselves are the main driver to such improvements. The role of tests on the effectiveness of TF has been studied in a controlled experiment by Erdogmus et al. (i.e. original study). Aim: Our goal is to examine the impact of test-first (TF) development on product quality and developer productivity, specifically the role that tests play in it. Method: We replicated the original study's controlled experiment by comparing an experimental group applying TF to a control group applying a test-last approach. We then carried out a correlation study in order to understand whether the number of tests is a good predictor for external quality and/or productivity. Results: Mann-Whitney tests did not show any significant difference between the two groups in terms of number of tests written (W=114.5, p=0.38), developers' productivity (W=90, p=0.82) and external quality (W=81.55, p=0.53). In addition, while a significant correlation exists between the number of tests and productivity (Spearman's ? = 0.57, p

Keywords

  • experiment replication
  • productivity
  • software quality
  • test-driven development

Cite this

@article{4cfeb1ccdece4c5daf3759a12ae47f7c,
title = "A replicated experiment on the effectiveness of test-first development",
abstract = "Background: Test-first development (TF) is regarded as a development practice that can lead to better quality of software products, as well as improved developer productivity. By implementing unit tests before the corresponding production code, the tests themselves are the main driver to such improvements. The role of tests on the effectiveness of TF has been studied in a controlled experiment by Erdogmus et al. (i.e. original study). Aim: Our goal is to examine the impact of test-first (TF) development on product quality and developer productivity, specifically the role that tests play in it. Method: We replicated the original study's controlled experiment by comparing an experimental group applying TF to a control group applying a test-last approach. We then carried out a correlation study in order to understand whether the number of tests is a good predictor for external quality and/or productivity. Results: Mann-Whitney tests did not show any significant difference between the two groups in terms of number of tests written (W=114.5, p=0.38), developers' productivity (W=90, p=0.82) and external quality (W=81.55, p=0.53). In addition, while a significant correlation exists between the number of tests and productivity (Spearman's ? = 0.57, p",
keywords = "experiment replication, productivity, software quality, test-driven development",
author = "Davide Fucci and Burak Turhan",
year = "2013",
month = "12",
day = "1",
doi = "10.1109/ESEM.2013.15",
language = "English",
pages = "103--112",
journal = "International Symposium on Empirical Software Engineering and Measurement",
issn = "1949-3770",

}

A replicated experiment on the effectiveness of test-first development. / Fucci, Davide; Turhan, Burak.

In: International Symposium on Empirical Software Engineering and Measurement, 01.12.2013, p. 103-112.

Research output: Contribution to journalConference articleResearchpeer-review

TY - JOUR

T1 - A replicated experiment on the effectiveness of test-first development

AU - Fucci, Davide

AU - Turhan, Burak

PY - 2013/12/1

Y1 - 2013/12/1

N2 - Background: Test-first development (TF) is regarded as a development practice that can lead to better quality of software products, as well as improved developer productivity. By implementing unit tests before the corresponding production code, the tests themselves are the main driver to such improvements. The role of tests on the effectiveness of TF has been studied in a controlled experiment by Erdogmus et al. (i.e. original study). Aim: Our goal is to examine the impact of test-first (TF) development on product quality and developer productivity, specifically the role that tests play in it. Method: We replicated the original study's controlled experiment by comparing an experimental group applying TF to a control group applying a test-last approach. We then carried out a correlation study in order to understand whether the number of tests is a good predictor for external quality and/or productivity. Results: Mann-Whitney tests did not show any significant difference between the two groups in terms of number of tests written (W=114.5, p=0.38), developers' productivity (W=90, p=0.82) and external quality (W=81.55, p=0.53). In addition, while a significant correlation exists between the number of tests and productivity (Spearman's ? = 0.57, p

AB - Background: Test-first development (TF) is regarded as a development practice that can lead to better quality of software products, as well as improved developer productivity. By implementing unit tests before the corresponding production code, the tests themselves are the main driver to such improvements. The role of tests on the effectiveness of TF has been studied in a controlled experiment by Erdogmus et al. (i.e. original study). Aim: Our goal is to examine the impact of test-first (TF) development on product quality and developer productivity, specifically the role that tests play in it. Method: We replicated the original study's controlled experiment by comparing an experimental group applying TF to a control group applying a test-last approach. We then carried out a correlation study in order to understand whether the number of tests is a good predictor for external quality and/or productivity. Results: Mann-Whitney tests did not show any significant difference between the two groups in terms of number of tests written (W=114.5, p=0.38), developers' productivity (W=90, p=0.82) and external quality (W=81.55, p=0.53). In addition, while a significant correlation exists between the number of tests and productivity (Spearman's ? = 0.57, p

KW - experiment replication

KW - productivity

KW - software quality

KW - test-driven development

UR - http://www.scopus.com/inward/record.url?scp=84893312566&partnerID=8YFLogxK

U2 - 10.1109/ESEM.2013.15

DO - 10.1109/ESEM.2013.15

M3 - Conference article

SP - 103

EP - 112

JO - International Symposium on Empirical Software Engineering and Measurement

T2 - International Symposium on Empirical Software Engineering and Measurement

JF - International Symposium on Empirical Software Engineering and Measurement

SN - 1949-3770

M1 - 6681343

ER -