Association for the
Advancement of Computing in Education
Celebrating 30+ Years of
Service to the IT in Education / E-Learning Community
Celebrating 30+ Years of
Service to the IT in Education / E-Learning Community
Invited SpeakerA review of EdMedia proceedings will reveal that there are diverse approaches to e-learning research across geographical and philosophical boundaries. Individual papers reveal work which is derived from particular paradigmatic and methodological perspectives. A question to ask is whether all of these approaches are valid, and whether they are equally valuable? The thesis of this presentation is that we need to look beyond methodologies to understand the nature of evaluation and research into e-learning.
This session reports on recent work that attempts a fundamental analysis of the nature of research into e-learning. It will start by clarifying some terms which are often used imprecisely: e-learning, evaluation, and research. It will then unpack the idea that e-learning development is a design activity which follows a lifecycle, and one might want to ask different questions at different stages of that lifecycle. Some of these questions might be judgment questions, while others might be understanding questions, which leads to the understanding that studies of e-learning involve a variable mixture of evaluation and research, and we use the term ‘evaluation research’ to capture this idea.
The presentation will discuss e-learning evaluation research in the context of different disciplinary and interdisciplinary research approaches, recognizing that we need to choose the approach which is most appropriate to the goal of the inquiry. To assist us with this, we will delve into the phenomenon of e-learning, arguing that e-learning is an artificial phenomenon, and that research approaches need to be cognizant of the design elements in e-learning, and the cyclical nature of e-learning development. The session will also argue that e-learning evaluation research involves a varying mixture of a ‘search for fundamental understanding’ and ‘consideration of use’.
We use these considerations to discuss the role of theory in research, and, in particular, in e-learning evaluation research, before applying the preceding arguments to the e-learning lifecycle, identifying five different forms of evaluation research: Baseline analysis, Design evaluation, Project-management evaluation, Formative evaluation, and Effectiveness Research.
The presentation will then turn its attention to practical matters – how to design an e-learning evaluation-research study. The design of such a study should take into account the cyclical nature of the e-learning design and development process, and a cyclical research approach is therefore appropriate. We recognize the strength of design-based research in this context, without claiming that it is appropriate in all circumstances. Various evaluation-research approaches may be appropriate at different stages of the e-learning lifecycle.
The presentation will conclude by unpacking the process of conducting evaluation research, and highlight the use of evaluation-research matrices to break down the complexity of an e-learning evaluation-research study through divide and conquer techniques. Together with the five different forms of evaluation research, we can conceptualise a number of questions which are appropriate at a particular position in the e-learning life cycle and then develop instruments to obtain evidence to address these questions.
Associate Professor Rob Phillips works in the Educational Development Unit at Murdoch University, Perth, Western Australia. He has worked as a researcher and academic at universities in Australia and Germany since 1982, with a background in theoretical chemistry and computer science. He has worked with educational technology since 1992, designing and project managing educational technology development projects across most discipline areas. His expertise includes educational design, e-learning, distance education, educational policy development and academic staff development. Currently, he provides professional development and mentoring for academic staff in the scholarship of learning and teaching.
His research interests include university policy issues; evaluation research in e-learning; making creative and innovative use of technology; learning analytics; and project management in educational innovations. Rob has 124 publications, including 50 refereed papers, and is active in the management of two journals: the Australasian Journal of Educational Technology and Research in Learning Technology. He has been principal author of two internationally published books “Developer’s guide to interactive multimedia” and “Evaluating e-learning: Guiding research and practice”.
He is a life member and past-president (1996-2000) of the Australasian Society for Computers in Learning in Tertiary Education (ascilite) and is a fellow of the Higher Education Research and Development Society of Australasia. Rob received a Citation for Outstanding Contribution to Student Learning from the Australian Learning and Teaching Council in 2007. He was an executive member of the Australasian Council on Open, Distance and E-learning (ACODE) from 2004-2006 and resumes this role in 2012.
