WRITING TO LEARN WITH AUTOMATED FEEDBACK THROUGH (LSA) LATENT SEMANTIC ANALYSIS: EXPERIENCES DEALING WITH DIVERSITY IN LARGE ONLINE COURSES

摘要

The increasing demand for higher education and life-long training has induced a raising supply of online courses provided both by distance education institutions and conventional face to face universities. Simultaneously, public universities' budgets have been experiencing serious cuts, at least in Europe. Due to this shortage of human and material resources, large online courses usually face great challenges to provide an extremely diverse student community with quality formative assessment, specially, the kind that offers rich and personalized feedback. Peer to peer assessment could partially address the problem, but involves its own shortcomings. Writing to learn (WTL) is a way to foster critical thinking and a suitable method to train soft skills such as analysis and synthesis abilities. These skills are the base for other complex learning methodologies such as PBL, case method, etc. WTL approach requires a regular feedback given by dedicated lecturers. Consistent assessing of free-text answers is more difficult than we usually assume, specially, when addressing large or massive courses. Using multiple choice 'objective' assessment appears an obvious alternative. However, the authors feel that this alternative shows serious shortcomings when aiming to produce outcomes based on written expression and complex analysis. To face this dilemma, the authors decided to test an LSA-based automatic assessment tool developed by researchers of Developmental and Educational Psychology Department at UNED (Spanish National Distance Education University) named G-Rubric. The experience was launched in 2014-2015. By using GRubric, we provided automated formative and iterative feedback to our students for their open-ended (70-200 words). Some conclusions can be drawn from our experience: 1 Automated-assessment software such as Gallito-G-Rubric is currently mature enough to be used with students obtaining quite satisfactory results in terms of acceptable accuracy. 2 This kind of systems is particularly apt and useful for on-line teaching, especially in massive courses such as MOOC. Nevertheless, they show also great potential for face-to-face or mixed teaching at any level. 3 The experience of adapting such a system to assess free-text short-answer questions to Economic History proved reasonably affordable in terms of time and effort invested. The trial's results seem to point out that interacting with G-Rubric can improve learning by giving detailed feedback: (a) encourages devoting more time to the task; (b) increases 'earnings' in the quality of answers; (c) increases motivation to work on activities (d) helps students to achieve better final answers. In this sense, it may soon become a viable tool for formative assessment. In the near future, automated assessment systems will be part of the teacher's toolbox, as Virtual Learning Environments are today. LSA-based systems such as G-Rubric are a solid candidate to a leading role in that process.