Effect of code coverage on software reliability measurement

摘要

Existing software reliability-growth models often over-estimatenthe reliability of a given program. Empirical studies suggest that thenover-estimations exist because the models do not account for the naturenof the testing. Every testing technique has a limit to its ability tonreveal faults in a given system. Thus, as testing continues in itsnregion of saturation, no more faults are discovered and inaccuratenreliability-growth phenomena are predicted from the models. This papernpresents a technique intended to solve this problem, using both time andncode coverage measures for the prediction of software failures innoperation. Coverage information collected during testing is used only tonconsider the effective portion of the test data. Execution time betweenntest cases, which neither increases code coverage nor causes a failure,nis reduced by a parameterized factor. Experiments were conducted tonevaluate this technique, on a program created in a simulated environmentnwith simulated faults, and on two industrial systems that containedntenths of ordinary faults. Two well-known reliability models,nGoel-Okumoto and Musa-Okumoto, were applied to both the raw data and tonthe data adjusted using this technique. Results show thatnover-estimation of reliability is properly corrected in the casesnstudied. This new approach has potential, not only to achieve morenaccurate applications of software reliability models, but to revealneffective ways of conducting software testing