LESSONS LEARNED AND MURPHY'S COROLLARY

摘要

Safety in flight test requires that when the potential consequences of a negative outcome are catastrophic, the go decision must be made only when a very high degree of confidence in a positive outcome exists. When this protocol is followed, it is a natural result that a very high percentage of outcomes will be positive, and it is further likely that we will see a long string of consecutive positive outcomes. When this happens a feedback loop is created in which the go decision is validated by the positive outcome as having been a good decision, and is therefore reinforced and made more likely to be made again in the future under similar circumstances. Beyond that, often the lesson learned from a series of consecutive positive outcomes is that our evaluative criteria for making the go decision were unnecessarily conservative; not only were we correct to make the go decision, but we could have made it under even somewhat less favorable circumstances. This can lead to a gradual lowering of our standards. However, consecutive successful outcomes can mask go decisions that may be fundamentally unsound, if the success is the result, either partly or entirely, of good fortune. Below a certain threshold of probability for success, repeated go decisions will lead eventually to an unacceptable probability of failure. At 99% probability of success, eighteen events results in a cumulative probability of failure for at least one event of 17%, or one chance in six. The common idiomatic statement of Murphy's Law is: "Anything that can go wrong will go wrong." This is likely not what Murphy said, nor is it the best statement of the lesson learned from Murphy's experience, and in fact, no one who thinks about it can actually believe it. Almost anything could go wrong, and if it were really true that anything that could go wrong would go wrong, then most everything would go wrong, and most things don't. But if Murphy's Law were true, then a corollary of this "law" would also be true: "Anything that did not go wrong could not have gone wrong." And while we don't really believe Murphy's Law, we are surprisingly susceptible to being seduced by an unconscious belief in Murphy's corollary - that is, we take a positive outcome as evidence that a positive outcome was inevitable. Combining with our failure to appreciate the nature of probability and our susceptibility to Murphy's corollary is the inevitable and powerful pressure we often experience to make the go decision under less than favorable circumstances. This pressure is a natural and unavoidable consequence of project deadlines, budgets, and performance expectations. It induces us to make the go decision when we otherwise might not, and to validate the go decision afterwards when the outcome is successful, when that validation may be inappropriate. A means for combating the effects of these three factors is to conduct a rigorous and quantitative pre and post-test analysis not only of failed outcomes, but also of successful ones, with the goal of identifying apparently successful outcomes that may have followed bad decisions, and that were successful because of good fortune.