A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?

Davide Fucci, Hakan Erdogmus, Burak Turhan, Markku Oivo, Natalia Juristo

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness. Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.

LanguageEnglish
Article number7592412
Pages597-614
Number of pages18
JournalIEEE Transactions on Software Engineering
Volume43
Issue number7
DOIs
Publication statusPublished - 1 Jul 2017
Externally publishedYes

Keywords

  • empirical investigation
  • external quality
  • process dimensions
  • productivity
  • Test-driven development

Cite this

Fucci, Davide ; Erdogmus, Hakan ; Turhan, Burak ; Oivo, Markku ; Juristo, Natalia. / A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?. In: IEEE Transactions on Software Engineering. 2017 ; Vol. 43, No. 7. pp. 597-614.
@article{d2059e34ea524b3c8b3b7a543fefaefa,
title = "A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?",
abstract = "Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness. Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.",
keywords = "empirical investigation, external quality, process dimensions, productivity, Test-driven development",
author = "Davide Fucci and Hakan Erdogmus and Burak Turhan and Markku Oivo and Natalia Juristo",
year = "2017",
month = "7",
day = "1",
doi = "10.1109/TSE.2016.2616877",
language = "English",
volume = "43",
pages = "597--614",
journal = "IEEE Transactions on Software Engineering",
issn = "0098-5589",
publisher = "Publ by IEEE",
number = "7",

}

A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last? / Fucci, Davide; Erdogmus, Hakan; Turhan, Burak; Oivo, Markku; Juristo, Natalia.

In: IEEE Transactions on Software Engineering, Vol. 43, No. 7, 7592412, 01.07.2017, p. 597-614.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?

AU - Fucci, Davide

AU - Erdogmus, Hakan

AU - Turhan, Burak

AU - Oivo, Markku

AU - Juristo, Natalia

PY - 2017/7/1

Y1 - 2017/7/1

N2 - Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness. Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.

AB - Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness. Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.

KW - empirical investigation

KW - external quality

KW - process dimensions

KW - productivity

KW - Test-driven development

UR - http://www.scopus.com/inward/record.url?scp=85029211132&partnerID=8YFLogxK

U2 - 10.1109/TSE.2016.2616877

DO - 10.1109/TSE.2016.2616877

M3 - Article

VL - 43

SP - 597

EP - 614

JO - IEEE Transactions on Software Engineering

T2 - IEEE Transactions on Software Engineering

JF - IEEE Transactions on Software Engineering

SN - 0098-5589

IS - 7

M1 - 7592412

ER -