Design Fractals: A Graphical Method for Evaluating Binary Covering Arrays

摘要

Test engineers are often faced with the challenge of selecting test cases that maximize the chance of discovering faults while working within a limited budget. Combinatorial testing is an effective test case selection strategy to address this challenge. The basic idea is to select test cases that ensure that all possible combinations of settings from two (or more) inputs are accounted for, regardless of which subset of inputs are selected. This is accomplished by using a covering array as the test case selection mechanism. However, for any given testing scenario, there are usually several alternative covering arrays that may satisfy budgetary and coverage requirements. Unfortunately, it is often unclear how to choose from these alternatives. Practitioners often want a way to explore how the input space is being covered to decide which alternative is best suited for the testing scenario. In this paper we propose a graphical method that may be used to evaluate binary covering arrays. The method allows practitioners to visualize input space coverage for inputs of interest. We illustrate the method with several examples.