Detecting systematic discrepancies in nursing home assessments of residents

摘要

Medicare and Medicaid reimbursement received by nursing homes are linked to the care needs of residents in a facility. Thus, a facility may have an incentive to overestimate the care needs of residents. To evaluate this, a sample of nursing homes in Ohio was selected, and independent assessors were sent to sampled facilities, and the rating of residents by the independent assessor was compared to a similar rating by the staff at nursing homes. We first evaluated whether the independent assessors were homogeneous with respect to agreement with nursing facility assessors using a permutation test procedure. Since there was evidence of heterogeneity among the independent assessors, we evaluated agreement between the independent nurse and facility assessors separately for each independent nurse. Responses were collapsed into three categories: independent assessor was lower, equal or greater than the facility assessor. Under a null hypothesis that lower and greater categories were equally probable, the maximum likelihood estimate of the vector of probabilities corresponding to these categories was constructed for each independent assessor. The P-value for detecting a difference between independent and facility assessors was calculated for each facility by summing the multinomial probabilities of obtaining a result at least as extreme as observed in the direction of the “lower” category being more probable. Five of the 39 facilities had P-value < 0.10 with three of the low P-value facilities occurring in reviews by one of the independent assessors. We believe that there is not evidence of systematic differences between the facility assessors and independent assessors although a caveat associated with this statement is the power of this assessment procedure is strongly linked to number of cases reviewed per facility along with the size of the effect expected. We believe this provides a screening mechanism to identify facilities where additional samples are warranted.