Using Item Parameters to Investigate Score Inflation.

摘要

Recent calls to hold schools and teachers accountable for student performance have raised the stakes on standardized testing. One possible response to such pressures is that teachers may narrow the curriculum or may coach students on specific item. Such behaviors can lead to score inflation, where inferences drawn about student performance do not accurately reflect students' knowledge.;A growing literature seeks to identify instances of score inflation, typically by comparing performances on two tests. In this thesis, I present a new method to identify a specific form of score inflation using information embedded in a single test. This method hinges on identifying systematic decreases in item discrimination parameters over time (which I call Discrimination Parameter Drift, or DPD). DPD could signal that teachers are focusing instruction on certain content standards or are coaching towards specific item types.;I demonstrate this method's effectiveness using three simulations. The discrimination of the simulated "coached" questions declines significantly over time while that of the noncoached items remains stable. I apply this method to datasets from two urban school districts. Using a combination of a three-parameter logistic model and Samejima's graded response model, I estimate item parameters in each grade and year and look for changes over time in average item discrimination within content standard. In District A, which shows little evidence of overall score inflation using traditional methods, I find suggestive evidence of DPD across standards. By contrast, in District B, which appears to have more substantial overall score inflation, I find no consistent evidence of DPD. Thus, my approach appears to identify a new pattern of score inflation, in some ways distinct from patterns uncovered by previous methods.;If DPD were the result of coaching, estimates of teacher effectiveness from test scores may be invalid. As a result, I estimate teacher-level value-added models in District A using three different sets of items: those with no evidence of DPD, those that did show evidence of DPD, and all items. Correlations between value-added estimates generated using scores purged of DPD-sensitive items and those using all items range only between 0.68 and 0.73.