Researcher Bias in software engineering experiments: a qualitative investigation

Simone Romano, Davide Fucci, Giuseppe Scanniello, Maria Teresa Baldassarre, Burak Turhan, Natalia Juristo

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

3 Citations (Scopus)


Researcher Bias (RB) occurs when researchers influence the results of an empirical study based on their expectations. RB might be due to the use of Questionable Research Practices (QRPs). In research fields like medicine, blinding techniques have been applied to counteract RB. We conducted an explorative qualitative survey to investigate RB in Software Engineering (SE) experiments, with respect to: (i) QRPs potentially leading to RB, (ii) causes behind RB, and (iii) possible actions to counteract RB including blinding techniques. Data collection was based on semi-structured interviews. We interviewed nine active experts in the empirical SE community. We then analyzed the transcripts of these interviews through thematic analysis. We found that some QRPs are acceptable in certain cases. Also, it appears that the presence of RB is perceived in SE and, to counteract RB, a number of solutions have been highlighted: some are intended for SE researchers and others for the boards of SE research outlets.

Original languageEnglish
Title of host publicationProceedings - 46th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2020
EditorsAntonio Martini, Manuel Wimmer, Amund Skavhaug
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages8
ISBN (Electronic)9781728195322
ISBN (Print)9781728195339
Publication statusPublished - 2020
EventEuromicro Conference on Software Engineering and Advanced Applications 2020 - Kranj, Slovenia
Duration: 26 Aug 202028 Aug 2020
Conference number: 46th (Proceedings) (Website)


ConferenceEuromicro Conference on Software Engineering and Advanced Applications 2020
Abbreviated titleSEAA 2020
Internet address


  • blinding
  • interview
  • researcher bias
  • Survey

Cite this