Effect of time-pressure on perceived and actual performance in functional software testing

Iflaah Salman; Burak Turhan

doi:10.1145/3202710.3203148

Effect of time-pressure on perceived and actual performance in functional software testing

Iflaah Salman
, Burak Turhan

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

8 Citations (Scopus)

Abstract

Background: Time-pressure is an inevitable reality of software industry that influences the performance of software engineers. It may result in adverse effects on software quality or distort the perception of performance on executed tasks to differ from actual performance.

Objective: We aim to investigate the effect of timepressure on perceived and actual performance of software testers in the context of functional software testing.

Method: We performed two controlled experiments with 87 graduate students in two academic terms. We assessed actual performance in terms of coverage (i.e. percentage of test cases correctly identified) and perceived performance using NASA-TLX. We have an independent factorial design for our experimental study.

Results: The results reveal a significant effect of time-pressure on actual performance. However, we could not observe a significant effect of time-pressure on the perceived performance of the participants for the task undertaken. We also observed a significant negative correlation between actual and perceived performance when controlled for time-pressure and experimental session factors.

Conclusion: Time-pressure affects the actual performance in a testing task but the perception of accomplishment by the testers is sustained irrespective of time-pressure, indicating an over-estimation issue. Perception of performance should be adjusted to align with reality to account for the effect of time pressure. This will lead to better self estimates of performance.

Original language	English
Title of host publication	ICSSP 2018 - May 26-27 – Gothenburg, Sweden
Subtitle of host publication	Proceedings of the 2018 International Conference on Software and System Process
Editors	Marco Kuhrmann, Rory V. O’Connor, Dan Houston, Regina Hebig
Place of Publication	New York NY USA
Publisher	Association for Computing Machinery (ACM)
Pages	130-139
Number of pages	10
ISBN (Electronic)	9781450364591
DOIs	https://doi.org/10.1145/3202710.3203148
Publication status	Published - 2018
Externally published	Yes
Event	International Conference on Software and System Process 2018 - Gothenburg, Sweden Duration: 26 May 2018 → 27 May 2018 https://dl.acm.org/citation.cfm?id=3202710&picked=prox (Proceedings)

Conference

Conference	International Conference on Software and System Process 2018
Abbreviated title	ICSSP 2018
Country/Territory	Sweden
City	Gothenburg
Period	26/05/18 → 27/05/18
Internet address	https://dl.acm.org/citation.cfm?id=3202710&picked=prox (Proceedings)

Keywords

Experiment
Performance
Self-assessment
Software Quality
Software Testing
Time Pressure

Access to Document

10.1145/3202710.3203148

Cite this

Salman, I., & Turhan, B. (2018). Effect of time-pressure on perceived and actual performance in functional software testing. In M. Kuhrmann, R. V. O’Connor, D. Houston, & R. Hebig (Eds.), ICSSP 2018 - May 26-27 – Gothenburg, Sweden: Proceedings of the 2018 International Conference on Software and System Process (pp. 130-139). Association for Computing Machinery (ACM). https://doi.org/10.1145/3202710.3203148

Salman, Iflaah ; Turhan, Burak. / Effect of time-pressure on perceived and actual performance in functional software testing. ICSSP 2018 - May 26-27 – Gothenburg, Sweden: Proceedings of the 2018 International Conference on Software and System Process. editor / Marco Kuhrmann ; Rory V. O’Connor ; Dan Houston ; Regina Hebig. New York NY USA : Association for Computing Machinery (ACM), 2018. pp. 130-139

@inproceedings{de6e21bee43c40ca9b69e9a7633569a7,

title = "Effect of time-pressure on perceived and actual performance in functional software testing",

abstract = "Background: Time-pressure is an inevitable reality of software industry that influences the performance of software engineers. It may result in adverse effects on software quality or distort the perception of performance on executed tasks to differ from actual performance. Objective: We aim to investigate the effect of timepressure on perceived and actual performance of software testers in the context of functional software testing. Method: We performed two controlled experiments with 87 graduate students in two academic terms. We assessed actual performance in terms of coverage (i.e. percentage of test cases correctly identified) and perceived performance using NASA-TLX. We have an independent factorial design for our experimental study. Results: The results reveal a significant effect of time-pressure on actual performance. However, we could not observe a significant effect of time-pressure on the perceived performance of the participants for the task undertaken. We also observed a significant negative correlation between actual and perceived performance when controlled for time-pressure and experimental session factors. Conclusion: Time-pressure affects the actual performance in a testing task but the perception of accomplishment by the testers is sustained irrespective of time-pressure, indicating an over-estimation issue. Perception of performance should be adjusted to align with reality to account for the effect of time pressure. This will lead to better self estimates of performance.",

keywords = "Experiment, Performance, Self-assessment, Software Quality, Software Testing, Time Pressure",

author = "Iflaah Salman and Burak Turhan",

year = "2018",

doi = "10.1145/3202710.3203148",

language = "English",

pages = "130--139",

editor = "Kuhrmann, \{Marco \} and \{V. O{\textquoteright}Connor\}, \{Rory \} and Houston, \{Dan \} and Hebig, \{Regina \}",

booktitle = "ICSSP 2018 - May 26-27 – Gothenburg, Sweden",

publisher = "Association for Computing Machinery (ACM)",

address = "United States of America",

note = "International Conference on Software and System Process 2018, ICSSP 2018 ; Conference date: 26-05-2018 Through 27-05-2018",

url = "https://dl.acm.org/citation.cfm?id=3202710\&picked=prox",

}

Salman, I & Turhan, B 2018, Effect of time-pressure on perceived and actual performance in functional software testing. in M Kuhrmann, R V. O’Connor, D Houston & R Hebig (eds), ICSSP 2018 - May 26-27 – Gothenburg, Sweden: Proceedings of the 2018 International Conference on Software and System Process. Association for Computing Machinery (ACM), New York NY USA, pp. 130-139, International Conference on Software and System Process 2018, Gothenburg, Sweden, 26/05/18. https://doi.org/10.1145/3202710.3203148

Effect of time-pressure on perceived and actual performance in functional software testing. / Salman, Iflaah; Turhan, Burak.
ICSSP 2018 - May 26-27 – Gothenburg, Sweden: Proceedings of the 2018 International Conference on Software and System Process. ed. / Marco Kuhrmann; Rory V. O’Connor; Dan Houston; Regina Hebig. New York NY USA: Association for Computing Machinery (ACM), 2018. p. 130-139.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Effect of time-pressure on perceived and actual performance in functional software testing

AU - Salman, Iflaah

AU - Turhan, Burak

PY - 2018

Y1 - 2018

N2 - Background: Time-pressure is an inevitable reality of software industry that influences the performance of software engineers. It may result in adverse effects on software quality or distort the perception of performance on executed tasks to differ from actual performance. Objective: We aim to investigate the effect of timepressure on perceived and actual performance of software testers in the context of functional software testing. Method: We performed two controlled experiments with 87 graduate students in two academic terms. We assessed actual performance in terms of coverage (i.e. percentage of test cases correctly identified) and perceived performance using NASA-TLX. We have an independent factorial design for our experimental study. Results: The results reveal a significant effect of time-pressure on actual performance. However, we could not observe a significant effect of time-pressure on the perceived performance of the participants for the task undertaken. We also observed a significant negative correlation between actual and perceived performance when controlled for time-pressure and experimental session factors. Conclusion: Time-pressure affects the actual performance in a testing task but the perception of accomplishment by the testers is sustained irrespective of time-pressure, indicating an over-estimation issue. Perception of performance should be adjusted to align with reality to account for the effect of time pressure. This will lead to better self estimates of performance.

AB - Background: Time-pressure is an inevitable reality of software industry that influences the performance of software engineers. It may result in adverse effects on software quality or distort the perception of performance on executed tasks to differ from actual performance. Objective: We aim to investigate the effect of timepressure on perceived and actual performance of software testers in the context of functional software testing. Method: We performed two controlled experiments with 87 graduate students in two academic terms. We assessed actual performance in terms of coverage (i.e. percentage of test cases correctly identified) and perceived performance using NASA-TLX. We have an independent factorial design for our experimental study. Results: The results reveal a significant effect of time-pressure on actual performance. However, we could not observe a significant effect of time-pressure on the perceived performance of the participants for the task undertaken. We also observed a significant negative correlation between actual and perceived performance when controlled for time-pressure and experimental session factors. Conclusion: Time-pressure affects the actual performance in a testing task but the perception of accomplishment by the testers is sustained irrespective of time-pressure, indicating an over-estimation issue. Perception of performance should be adjusted to align with reality to account for the effect of time pressure. This will lead to better self estimates of performance.

KW - Experiment

KW - Performance

KW - Self-assessment

KW - Software Quality

KW - Software Testing

KW - Time Pressure

UR - https://www.scopus.com/pages/publications/85048368743

U2 - 10.1145/3202710.3203148

DO - 10.1145/3202710.3203148

M3 - Conference Paper

AN - SCOPUS:85048368743

SP - 130

EP - 139

BT - ICSSP 2018 - May 26-27 – Gothenburg, Sweden

A2 - Kuhrmann, Marco

A2 - V. O’Connor, Rory

A2 - Houston, Dan

A2 - Hebig, Regina

PB - Association for Computing Machinery (ACM)

CY - New York NY USA

T2 - International Conference on Software and System Process 2018

Y2 - 26 May 2018 through 27 May 2018

ER -

Salman I, Turhan B. Effect of time-pressure on perceived and actual performance in functional software testing. In Kuhrmann M, V. O’Connor R, Houston D, Hebig R, editors, ICSSP 2018 - May 26-27 – Gothenburg, Sweden: Proceedings of the 2018 International Conference on Software and System Process. New York NY USA: Association for Computing Machinery (ACM). 2018. p. 130-139 doi: 10.1145/3202710.3203148

Effect of time-pressure on perceived and actual performance in functional software testing

Abstract

Conference

Keywords

Access to Document

Other files and links

Cite this