Poster: How developers debug software - The DBGBENCH dataset

Marcel Bohme, Ezekiel O. Soremekun, Sudipta Chattopadhyay, Emamurho Juliet Ugherughe, Andreas Zeller

Research output: Chapter in Book/Report/Conference proceedingConference PaperOtherpeer-review

4 Citations (Scopus)


How do professional software engineers debug computer programs? In an experiment with 27 real bugs that existed in several widely used programs, we invited 12 professional software engineers, who together spent one month on localizing, explaining, and fixing these bugs. This did not only allow us to study the various tools and strategies used to debug the same set of errors. We could also determine exactly which statements a developer would localize as faults, how a developer would diagnose and explain an error, and how a developer would fix an error - all of which software engineering researchers seek to automate. Until now, it has been difficult to evaluate the effectiveness and utility of automated debugging techniques without a user study. We publish the collected data, called DBGBENCH, to facilitate the effective evaluation of automated fault localization, diagnosis, and repair techniques w.r.t. The judgement of human experts.

Original languageEnglish
Title of host publicationProceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion - ICSE-C 2017
Subtitle of host publication20–28 May 2017 Buenos Aires, Argentina
EditorsAlessandro Orso, Martin Robillard
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages3
ISBN (Electronic)9781538615898
Publication statusPublished - 2017
Externally publishedYes
EventInternational Conference on Software Engineering 2017 - Buenos Aires, Argentina
Duration: 20 May 201728 May 2017
Conference number: 39th (Proceedings)


ConferenceInternational Conference on Software Engineering 2017
Abbreviated titleICSE 2017
CityBuenos Aires
OtherIEEE/ACM International Conference on Software Engineering Companion (ICSE-C 2017)
Internet address


  • Bug Diagnosis
  • Debugging in Practice
  • Fault Localization
  • Software Repair
  • Tool Evaluation
  • User as Tool Benchmark

Cite this