The tutor in the loop model for formative assessment

• Das Tutor-in-the-Loop-Modell für formatives Assessment

Herding, Daniel Chu; Schroeder, Ulrik (Thesis advisor)

Aachen : Publikationsserver der RWTH Aachen University (2013)
Dissertation / PhD Thesis

Aachen, Techn. Hochsch., Diss., 2013

Abstract

Closed exercise types like multiple-choice tests are widespread in higher education, especially for purposes of summative assessment. However, they cannot be considered learner-centric because they do not allow individual solutions. In contrast, assessing open-ended tasks is considerably more complex. Intelligent tutoring systems (ITS) aim at giving individual feedback fully autonomously, but due to their high development cost, they are available only for few domains. This thesis presents the tutor-in-the-loop model, which follows an alternative, semi-automatic approach to formative assessment. It combines automatic and tutorial feedback with the goal of letting the computer give feedback to standard solutions and typical mistakes. Students who need additional help or who have come up with extraordinary solutions can request feedback from their tutor. The students' interactions with the electronic exercises are recorded to help the tutor comprehend their solution processes. This log data can also be used to perform learning analytics: all logs of a course can be analyzed to detect common mistakes or to detect students who may need help. Various systems were developed to evaluate the tutor-in-the-loop model. A general framework was realized to cover aspects of automatic feedback which learning applications have in common. Several proofs of concept were implemented to show the applicability of the framework in different contexts. Finally, a logging component was realized based on an existing capture and replay toolkit. Evaluations showed intrinsic drawbacks of this toolkit in terms of deployment and analytics support. Therefore, it was replaced by a new system for logging and learning analytics. During evaluations it was found out that, while learners indeed access automatically generated feedback when necessary, some are reluctant to request feedback from their tutors. Privacy concerns were a major impediment to tutorial feedback requests. Still, several students contacted their tutors, and their questions could be answered with the help of the solution process logs.

Institutions

• Department of Computer Science [120000]
• The Learning Technologies Research Group [122420]