Experience report: an industrial experience report on test outsourcing practices

Xin Xia, David Lo, Pavneet Singh Kochhar, Zhenchang Xing, Xinyu Wang, Shanping Li

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

5 Citations (Scopus)


Nowadays, many companies contract their testing functionalities out to third-party IT outsourcing companies. This process referred to as test outsourcing is common in the industry, yet it is rarely studied in the research community. In this paper, to bridge the gap, we performed an empirical study on test outsourcing with 10 interviewees and 140 survey respondents. We investigated various research questions such as the types, the process, and the challenges of test outsourcing, and the differences between test outsourcing and in-house testing. We found customer satisfaction, tight project schedule, and domain unfamiliarity are the top-3 challenges faced by the testers. We also found there are substantial differences between test outsourcing and in-house testing. For example, most of the test outsourcing projects focused on functional test, and rarely did unit test. Also, due to privacy policies of client companies, test outsourcing is performed mainly on the binary distributions of the projects, and rarely the testers can touch the source code. Our findings have implications for future research. For instance, as a starting point, researchers can create automated program comprehension tools which work on binary distributions of projects to help testers better design effective test cases.

Original languageEnglish
Title of host publication2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE 2015)
Subtitle of host publicationGaithersburg, Maryland, USA 2 – 5 November 2015
EditorsHelene Waeselynck, Yvan Labiche
Place of PublicationWashington DC USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages11
ISBN (Electronic)9781509004065, 9781509004058
ISBN (Print)9781509004072
Publication statusPublished - 2015
Externally publishedYes
EventInternational Symposium on Software Reliability Engineering 2015 - Gaithersbury, United States of America
Duration: 2 Nov 20155 Nov 2015
Conference number: 26th


ConferenceInternational Symposium on Software Reliability Engineering 2015
Abbreviated titleISSRE 2015
Country/TerritoryUnited States of America
Internet address

Cite this