Conducting a rigorous quasi-experimental evaluation using a school district's existing student database

摘要

Often, when an evaluation is needed, there is always a tendency to jump into data collection without first considering what might be already available. This paper demonstrates how an existing student database from a large school district could be used to provide some rigorous evidence regarding the impact of a supplementary teacher professional development programme on student achievement. Randomization was not possible in this study but to ensure rigour, matching of the treatment and the comparison groups was done on variables that were considered highly predictive of future achievement. The two groups were matched at the class level on course type, grade level and prior achievement. Next, hierarchical linear modelling was used to further control for selection bias between the two groups. According to the statistical theory on selection bias, this two-part process of matching and modelling removes all the observed bias. This illustration shows that performing a rigorous quasi-experimental study is possible in education even when there are limiting circumstances to conducting a randomized experiment.