A cost-effective random testing method for programs with non-numeric inputs

Arlinta C. Barus, Tsong Yueh Chen, Fei-Ching Kuo, Huai Liu, Robert Merkel, Gregg Rothermel

    Research output: Contribution to journalArticleResearchpeer-review

    19 Citations (Scopus)

    Abstract

    Random testing (RT) has been widely used in the testing of various software and hardware systems. Adaptive random testing (ART) is a family of random testing techniques that aim to enhance the failure-detection effectiveness of RT by spreading random test cases evenly throughout the input domain. ART has been empirically shown to be effective on software with numeric inputs. However, there are two aspects of ART that need to be addressed to render its adoption more widespread - applicability to programs with non-numeric inputs, and the high computation overhead of many ART algorithms. We present a linear-order ART algorithm for software with non-numeric inputs. The key requirement for using ART with non-numeric inputs is an appropriate "distance" measure. We use the concepts of categories and choices from category-partition testing to formulate such a measure. We investigate the failure-detection effectiveness of our technique by performing an empirical study on 14 object programs, using two standard metrics - F-measure and P-measure. Our ART algorithm statistically significantly outperforms RT on 10 of the 14 programs studied, and exhibits performance similar to RT on three of the four remaining programs. The selection overhead of our ART algorithm is close to that of RT.
    Original languageEnglish
    Article number7442567
    Pages (from-to)3509-3523
    Number of pages15
    JournalIEEE Transactions on Computers
    Volume65
    Issue number12
    DOIs
    Publication statusPublished - 1 Dec 2016

    Keywords

    • Random testing
    • Adaptive random testing
    • Category-partition method

    Cite this