An industry experiment on the effects of test-driven development on external quality and productivity

Ayse Tosun, Oscar Dieste, Davide Fucci, Sira Vegas, Burak Turhan, Hakan Erdogmus, Adrian Santos, Markku Oivo, Kimmo Toro, Janne Jarvinen, Natalia Juristo

Research output: Contribution to journalArticleResearchpeer-review

6 Citations (Scopus)

Abstract

Existing empirical studies on test-driven development (TDD) report different conclusions about its effects on quality and productivity. Very few of those studies are experiments conducted with software professionals in industry. We aim to analyse the effects of TDD on the external quality of the work done and the productivity of developers in an industrial setting. We conducted an experiment with 24 professionals from three different sites of a software organization. We chose a repeated-measures design, and asked subjects to implement TDD and incremental test last development (ITLD) in two simple tasks and a realistic application close to real-life complexity. To analyse our findings, we applied a repeated-measures general linear model procedure and a linear mixed effects procedure. We did not observe a statistical difference between the quality of the work done by subjects in both treatments. We observed that the subjects are more productive when they implement TDD on a simple task compared to ITLD, but the productivity drops significantly when applying TDD to a complex brownfield task. So, the task complexity significantly obscured the effect of TDD. Further evidence is necessary to conclude whether TDD is better or worse than ITLD in terms of external quality and productivity in an industrial setting. We found that experimental factors such as selection of tasks could dominate the findings in TDD studies.

Original languageEnglish
Pages (from-to)2763-2805
Number of pages43
JournalEmpirical Software Engineering
Volume22
Issue number6
DOIs
Publication statusPublished - 2017
Externally publishedYes

Keywords

  • External quality
  • Industry experiment
  • Productivity
  • Test-driven development

Cite this

Tosun, A., Dieste, O., Fucci, D., Vegas, S., Turhan, B., Erdogmus, H., Santos, A., Oivo, M., Toro, K., Jarvinen, J., & Juristo, N. (2017). An industry experiment on the effects of test-driven development on external quality and productivity. Empirical Software Engineering, 22(6), 2763-2805. https://doi.org/10.1007/s10664-016-9490-0