Measurement System Analysis with Attribute Data

摘要

Every project captures defects which are classified based on multiple attributes like Severity, Defect Type, Injection & Detection Phase, and are used for Root Cause Analysis (RCA), based on which action items are derived to improve defect occurrence. However, we do not explore to see if different developers (who are also peer reviewers during reviews) who do this classification have same understanding of attributes. If the understanding is different among developers, the classified data used for RCA may provide a different picture & actions planned will not lead to improvement in defects thereby making it an ineffective RCA. Measurement System Analysis (MSA) is scientific and objective method of analyzing the validity of a measurement system for use by quantifying its accuracy, precision, and stability. Gage R&R, an MSA methodology widely used for attribute data was piloted in a project with the following sample: 1. 14 defects across Code Review, IT & ST which were classified by Subject Matter Experts 2. 6 developers with varying experiences across the project selected to classify these defects in 2 trials 3. The data was analyzed using Minitab tool in following steps: 3.1 Attribute Agreement Analysis 3.2 Comparing Appraisers against themselves 3.3 Comparing Appraisers against a known Standard 3.4 Comparing between Appraisers 3.5 Agreement between all Appraisers together and the Standard Gage R&R result needs be drawn based on overall Kappa values from the above 5 analysis steps. The result from sample study was that we need to improve the ability of developers to make better decision in categorizing the defects by providing training & bringing in more clarity on attributes. Attribute Gage R&R was successfully implemented for the first time in CT DD DS AA DF-PD. The study helped us to device trainings specific to individuals & also teams. We now intend to spread this best practice across projects within & outside DF-PD.