This paper presents the second phase of research about designing and testing the effect of systems simulations for building systems understanding. It builds on work presented last year in which we used a paired experiment in an introductory level college course to examine the effect of a simulation on understanding of simple accumulation principles. Previous results showed significant differences in some measures of understanding of systems principles but also highlighted issues with simulation design, comparability of subject groups, and measures of systems understanding. In this phase, we revised the simulation, learning measures, and study design. All students used the simulation. We compared the extent to which students interacted with the simulation with their performance on a set of systems thinking measures. Pre-test/ post-test measures showed strong improvement in understanding of accumulations principles among users who ran the simulations a moderate number of times, but analysis of the data shows it is not a linear relationship. Mid-range users (total run count between 10 and 20 runs on two different simulations) did significantly better than both low-range (1-9 runs) and high-range users (20+), indicating the simulations improved scores, but that there may be both a threshold and a saturation point in the effect of simulations on systems learning.
展开▼
