Background: Writing unit tests is one of the primary activities in test-driven development. Yet, the existing reviews report few evidence supporting or refuting the effect of this development approach on test case quality. Lack of ability and skills of developers to produce sufficiently good test cases are also reported as limitations of applying test-driven development in industrial practice.
Objective: We investigate the impact of test-driven development on the effectiveness of unit test cases compared to an incremental test last development in an industrial context.
Method: We conducted an experiment in an industrial setting with 24 professionals. Professionals followed the two development approaches to implement the tasks. We measure unit test effectiveness in terms of mutation score. We also measure branch and method coverage of test suites to compare our results with the literature.
Results: In terms of mutation score, we have found that the test cases written for a testdriven development task have a higher defect detection ability than test cases written for an incremental test-last development task. Subjects wrote test cases that cover more branches on a test-driven development task compared to the other task. However, test cases written for an incremental test-last development task cover more methods than those written for the second task.
Conclusion: Our findings are different from previous studies conducted at academic settings. Professionals were able to perform more effective unit testing with test-driven development. Furthermore, we observe that the coverage measure preferred in academic studies reveal different aspects of a development approach. Our results need to be validated in larger industrial contexts.
|Title of host publication||ICSSP 2018 - May 26-27 – Gothenburg, Sweden|
|Subtitle of host publication||Proceedings of the 2018 International Conference on Software and System Process|
|Editors||Marco Kuhrmann, Rory V. O’Connor, Dan Houston, Regina Hebig|
|Place of Publication||New York NY USA|
|Publisher||Association for Computing Machinery (ACM)|
|Number of pages||10|
|Publication status||Published - 2018|
|Event||International Conference on Software and System Process 2018 - Gothenburg, Sweden|
Duration: 26 May 2018 → 27 May 2018
|Conference||International Conference on Software and System Process 2018|
|Abbreviated title||ICSSP 2018|
|Period||26/05/18 → 27/05/18|
- Code coverage
- Empirical study
- Mutation score
- Test-driven development
- Unit testing