A controlled experiment with novice developers on the impact of task description granularity on software quality in Test-Driven Development

Background: Test-Driven Development (TDD) is an iterative software development process characterized by test-code-refactor cycle. TDD recommends that developers work on small and manageable tasks at each iteration. However, the ability to break tasks into small work items effectively is a learned skill that improves with experience. In experimental studies of TDD, the granularity of task descriptions is an overlooked factor. In particular, providing a more granular task description in terms of a set of sub-tasks versus providing a coarser-grained, generic description. Objective: We aim to investigate the impact of task description granularity on the outcome of TDD, as implemented by novice developers, with respect to software quality, as measured by functional correctness and functional completeness. Method: We conducted a one-factor crossover experiment with 48 graduate students in an academic environment. Each participant applied TDD and implemented two tasks, where one of the tasks was presented using a more granular task description. Resulting artifacts were evaluated with acceptance tests to assess functional correctness and functional completeness. Linear mixed-effects models (LMM) were used for analysis. Results: Software quality improved significantly when participants applied TDD using more granular task descriptions. The effect of task description granularity is statistically significant and had a medium to large effect size. Moreover, the task was found to be a significant predictor of software quality which is an interesting result (because two tasks used in the experiment were considered to be of similar complexity). Conclusion: For novice TDD practitioners, the outcome of TDD is highly coupled with the ability to break down the task into smaller parts. For researchers, task selection and task description granularity requires more attention in the design of TDD experiments. Task description granularity should be taken into account in secondary studies. Further comparative studies are needed to investigate whether task descriptions affect other development processes similarly.