An investigation into the structure of text characteristics and reader abilities in a test of second language reading.

摘要

This dissertation investigated the extent to which the Reading Comprehension section of the Test of English as a Foreign Language (TOEFL) (TOEFL Program Services, 1997) reflects current research about the nature of reading, and posed the following research questions: (1) What kinds of tasks are used in high-stakes EFL tests of reading, and how well do they reflect current thinking about the task variables involved in reading? (2) What do these tests appear to be measuring, in terms of the skills, abilities, processes, or other reader variables that they involve, and how well does this reflect current reading research about such variables? (3) What relationships exist between task characteristics and item response theory (IRT) item parameters for these tests?; These questions were answered using both qualitative and quantitative analyses. For Research Question 1, the task characteristics of the items and passages from three test forms were rated in a qualitative analysis. Treating items as the object of measurement, exploratory and confirmatory factor analyses (EFA and CFA) using structural equation modeling (SEM) were then used to construct and test a factor model of those task characteristics. This model contained 20 task characteristic variables describing both reading passages and the key sentences that examinees needed to comprehend to answer items correctly. Seven factors were identified: a passage content specificity factor, three key sentence syntactic features factors, three vocabulary factors, and two higher-order linguistic factors.; For Research Question 2, EFA and CFA were performed to pose and empirically test a factor model of the abilities measured by one form of the test. A general academic reading factor was found, along with two other factors involving physical sciences passage content and content from other academic disciplines.; For Research Question 3, an attempt was made to determine the amount of IRT item parameter variance explainable using the Phase 1 task characteristics model. Item discrimination had a small but significant loading on the passage content specificity factor. Surprisingly, no other item parameters loaded significantly on any of the task characteristic factors.; These findings were interpreted as having implications for language testing theory, research methodology, and practice.