## Abstract

We study the relative efficiencies of the random and systematic approaches to automated software testing. Using a simple but realistic set of assumptions, we propose a general model for software testing and define sampling strategies for random (R) and systematic (S_{0}) testing, where each sampling is associated with a sampling cost: 1 and c units of time, respectively. The two most important goals of software testing are: (i) achieving in minimal time a given degree of confidence x in a program's correctness and (ii) discovering a maximal number of errors within a given time bound n. For both (i) and (ii), we show that there exists a bound on c beyond which R performs better than S_{0} on the average. Moreover for (i), this bound depends asymptotically only on x. We also show that the efficiency of R can be fitted to the exponential curve. Using these results we design a hybrid strategy H that starts with R and switches to S_{0} when S_{0} is expected to discover more errors per unit time. In our experiments we find that H performs similarly or better than the most efficient of both and that S_{0} may need to be significantly faster than our bounds suggest to retain efficiency over R.

Original language | English |
---|---|

Article number | 7289448 |

Pages (from-to) | 345-360 |

Number of pages | 16 |

Journal | IEEE Transactions on Software Engineering |

Volume | 42 |

Issue number | 4 |

DOIs | |

Publication status | Published - Apr 2016 |

Externally published | Yes |

## Keywords

- Efficient Testing
- Error-based Partitioning
- Partition Testing
- Random Testing
- Testing Theory