ON CHANCE-ADJUSTED MEASURES FOR ACCURACY ASSESSMENT IN REMOTE SENSING IMAGE CLASSIFICATION

摘要

The underlying rationale and practical utility of chance-adjusted indices (e.g., kappa, tau) as accuracy measures in image classification have been under criticism for a long time despite the fact that they are near universally used. It has been suggested that the degree of chance agreement may be overestimated, or it makes no sense to use kappa or tau for their declared objectives due to the inconsistency of the chance definition. On the contrary, user's accuracy, producer's accuracy, and overall accuracy should be recommended because they are directly interpretable as probabilities of correct classification. Besides the continuing criticism in remote sensing literature, much more discussions can be found in psychology and sociology literature where kappa originated. In this paper, we give a review on literature of the chance-adjusted measures, specifically kappa-like measures. We focus our discussion on whether those measures are theoretically sound and practically interpretable. We re-evaluate the usefulness of kappa-like measures and give our recommendation of proper accuracy measures for accuracy assessment.