Vlatko Lukarov


Evaluating Learning Analytics in Higher Education

The overall purpose of this project is to improve Technology Enhanced Learning by introducing Learning Analytics tools on wide scale in higher education, namely at RWTH Aachen University. The purpose will be completed in several packages, each one covering different aspects one needs to consider when trying to develop and apply end-to-end process and methodology for effective Learning Analytics.

The first project package will explore and provide methods how to best capture the needs of the stakeholders - in our case, students, teaching staff, and the administration - in the learning processes. The results of this package will provide techniques to prioritize the stakeholders’ needs, while identifying opportunities for innovations in Learning Analytics based on the identified needs.

The second project package will examine how the Learning Analytics mechanisms and tools should be integrated into the blended learning processes. The outcome of this project package will provide the most appropriate location and position where should the Learning Analytics tools be placed, in order to meet the needs of the students, the teachers and the administration.

The third project package will provide empirical tools and methods to evaluate the Learning Analytics implementation in terms of effectiveness, and provide empirically sound evidence that the tools meet the needs of the stakeholders. The salient points of this project package are to develop evaluation criteria and quality standards for Learning Analytics, and inspect which are most suitable research evaluation methods for measuring impact and effectiveness of Learning Analytics tools.

The fourth project package is orthogonal to the previous project packages and it deals with the ethical and privacy dimensions in relation to Learning Analytics. Namely what needs to be considered in terms of ethics and privacy when applying Learning Analytics in higher education? The outcome of this package will cover different ethical and privacy dimensions with their conflicting areas, and provide practical solutions to resolve these conflicts. Furthermore, it will be examined how these ethical and privacy dimensions fit in with the needs of the stakeholders, and present the effects of introducing such tools as standard in higher education.

The project follows design-based research principles in order to provide answers to practical problems and research questions that arise in each of the project packages. The enactment of each project package will contribute empirically based methodology how to implement effective learning analytics tools, while considering users’ needs and goals, evaluation metrics and procedures for measuring effectiveness and impact, and ethics and privacy guidelines and solutions for the introduction of learning analytics in higher education.