An empirical study of bugs in software build systems

Xin Xia, Xiaozhen Zhou, David Lo, Xiaoqiong Zhao

Research output: Contribution to conferencePaperpeer-review

18 Citations (Scopus)


Build system converts source code, libraries and other data into executable programs by orchestrating the execution of compilers and other tools. The whole building process is managed by a software build system, such as Make, Ant, CMake, Maven, Scons, and QMake. The reliability of software build systems would affect the reliability of the build process. In this paper, we perform an empirical study on bugs in software build systems. We analyze four software build systems, Ant, Maven, CMake and QMake, which are four typical and widely-used software build systems, and can be used to build Java, C, C++ systems. We investigate their bug database and code repositories, randomly sample a set of bug reports and their fixes (800 bugs reports totally, and 199, 250, 200, and 151 bug reports for Ant, Maven, CMake and QMake, respectively), and manually assign them into various categories. We find that 21.35% of the bugs belong to the external interface category, 18.23% of the bugs belong to the logic category, and 12.86% of the bugs belong to the configuration category. We also investigate the relationship between bug categories and bug severities.

Original languageEnglish
Number of pages4
Publication statusPublished - 21 Oct 2013
Externally publishedYes
Event13th International Conference on Quality Software, QSIC 2013 - Nanjing, Jiangsu, China
Duration: 29 Jul 201330 Jul 2013


Conference13th International Conference on Quality Software, QSIC 2013
CityNanjing, Jiangsu


  • Bug Category
  • Empirical Study
  • Software Build System

Cite this