On the effectiveness of unit tests in test-driven development

Ayse Tosun, Muzamil Ahmed, Burak Turhan, Natalia Juristo

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

1 Citation (Scopus)

Abstract

Background: Writing unit tests is one of the primary activities in test-driven development. Yet, the existing reviews report few evidence supporting or refuting the effect of this development approach on test case quality. Lack of ability and skills of developers to produce sufficiently good test cases are also reported as limitations of applying test-driven development in industrial practice. 

Objective: We investigate the impact of test-driven development on the effectiveness of unit test cases compared to an incremental test last development in an industrial context. 

Method: We conducted an experiment in an industrial setting with 24 professionals. Professionals followed the two development approaches to implement the tasks. We measure unit test effectiveness in terms of mutation score. We also measure branch and method coverage of test suites to compare our results with the literature. 

Results: In terms of mutation score, we have found that the test cases written for a testdriven development task have a higher defect detection ability than test cases written for an incremental test-last development task. Subjects wrote test cases that cover more branches on a test-driven development task compared to the other task. However, test cases written for an incremental test-last development task cover more methods than those written for the second task. 

Conclusion: Our findings are different from previous studies conducted at academic settings. Professionals were able to perform more effective unit testing with test-driven development. Furthermore, we observe that the coverage measure preferred in academic studies reveal different aspects of a development approach. Our results need to be validated in larger industrial contexts.

Original languageEnglish
Title of host publicationICSSP 2018 - May 26-27 – Gothenburg, Sweden
Subtitle of host publicationProceedings of the 2018 International Conference on Software and System Process
EditorsMarco Kuhrmann, Rory V. O’Connor, Dan Houston, Regina Hebig
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages113-122
Number of pages10
ISBN (Electronic)9781450364591
DOIs
Publication statusPublished - 2018
Externally publishedYes
EventInternational Conference on Software and System Process 2018 - Gothenburg, Sweden
Duration: 26 May 201827 May 2018
https://dl.acm.org/citation.cfm?id=3202710&picked=prox (Proceedings)

Conference

ConferenceInternational Conference on Software and System Process 2018
Abbreviated titleICSSP 2018
CountrySweden
CityGothenburg
Period26/05/1827/05/18
Internet address

Keywords

  • Code coverage
  • Empirical study
  • Mutation score
  • Test-driven development
  • Unit testing

Cite this

Tosun, A., Ahmed, M., Turhan, B., & Juristo, N. (2018). On the effectiveness of unit tests in test-driven development. In M. Kuhrmann, R. V. O’Connor, D. Houston, & R. Hebig (Eds.), ICSSP 2018 - May 26-27 – Gothenburg, Sweden: Proceedings of the 2018 International Conference on Software and System Process (pp. 113-122). Association for Computing Machinery (ACM). https://doi.org/10.1145/3202710.3203153