Impact of process conformance on the effects of test-driven development

Davide Fucci, Burak Turhan, Markku Oivo

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

6 Citations (Scopus)


Context: One limitation of the empirical studies about test-driven development (TDD) is knowing whether the developers followed the advocated test-code-refactor cycle. Research dealt with the issue of process conformance only in terms of internal validity, while investigating the role of other confounding variables that might explain the controversial effects of TDD. None of the research included process conformance as a fundamental part of the analysis. Goal: We aim to examine the impact of process conformance on the claimed effects of TDD on external quality, developers' productivity and test quality. Method: We used data collected during a previous study to create regression models in which the level of process conformance was used to predict external quality, productivity, and tests thoroughness. Result: Based on our analysis of the available data (n = 22), we observe that neither quality (p - value = 0.21), productivity (p - value = 0.80), number of tests (p - value = 0.39) nor coverage (p - value = 0.09) was correlated with the level of TDD process conformance. Conclusion: While based on a small sample, we raise concerns about how TDD is interpreted. We also question whether the cost of strictly following TDD will pay-off in terms of external quality, productivity, and tests thoroughness.

Original languageEnglish
Title of host publication8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2014
Subtitle of host publication18th-19th September 2014 Torino, ITALY
EditorsTore Dybå, Marco Torchiano
Place of PublicationNew York NY USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages10
ISBN (Electronic)9781450327749
Publication statusPublished - 2014
Externally publishedYes
EventInternational Symposium on Empirical Software Engineering and Measurement 2014 - Torino, Italy
Duration: 18 Sep 201419 Sep 2014
Conference number: 8th


ConferenceInternational Symposium on Empirical Software Engineering and Measurement 2014
Abbreviated titleESEM 2014
Internet address


  • developers productivity
  • regression models
  • software quality
  • test-driven development

Cite this