Measurement Issues of Some Widely Used Instruments May Mask Positive Findings in Medical Research -With Illustration of the Hamilton Rating Scale for Depression (HRSD) in a Large Clinical Trial for Major Depression

摘要

Recent years have seen increasing popularity of subjective patient-reported outcomes (PRO) such as depression and quality of life in medical research. These subjective outcomes needed to be measured by some instruments such as the Hamilton Rating Scale for Depression (HRSD) and the Medical Outcomes Study 36-Item Short-Form Health Survey (SF-36). Unfortunately, most of instruments in medical research had not undergone rigorous development and evaluation stages (Teresi & Fleishman 2007). Here is an example. Many depression trials used the Hamilton Rating Scale for Depression (HRSD) as the primary measure of depression, although its severe measurement issues had been well-known. In a trail (Kellner, Kanpp, et al., 2006) that compares continuation electroconvulsive therapy (C-ECT) vs. Pharmacotherapy (C-Pharm) for relapse prevention in major depression, the total score of the 24-item HRSD (HRSD24) was used to define relapse or remission. That trail found no statistically significant differences between the two arms. However, the results might be misleading, simply because the unidimensional assumption made for the HRSD24, i.e., all of the 24 items are measuring a single domain (depression) and therefore the 24 item scores can be summed to a single total score as a measure of depression, might be wrong. In this study, the original data from that trial (201 patients with 98 C-ECT, 103 C-Pharm) were utilized to investigate measurement issues in the HRSD24. Confirmatory factor analyses (CFA) for the unidimensional assumption of the HRSD24 were implemented on the data at visits 1, 2, 3, and 4. Results showed that this uni-dimensional assumption failed at each of the 4 visits. This indicated that the single total HRSD24 score should not be used as the defining variable for relapse or remission at any of the visits, thus the analyses using this total score was misleading, and may conceal some true positive findings in that trial. Exploratory factor analysis (EFA) on all of the 24 items failed to yield a consistent factor structure across the 4 visits. Then item-level analyses of the 24 items were implemented, and results showed that statistically significant differences exist on 3 of the 24 items between the two arms. Clearly the measurement issues of the HRSD masked some positive findings in this large depression trial. Similarly, other widely used but not rigorously tested instruments may also masked some important findings in medical studies, especially those with negative findings.