Designing Assessment
Assessment plays a vital role in delivering, evaluating, monitoring, improving and shaping learning experiences on the Web, at the desk and in the classroom. In the process of orchestrating educational technologies instructional designers are often confronted with the challenge of designing or deploying creative and authentic assessment techniques.
Fostered by the rise of constructivist learning theory, authentic assessment, a.k.a. performance assessment as well as connected approaches and tools –such as rubrics, portfolios and competency-based learning outcomes –have been discussed in educational research since the mid-nineties. This paradigm shift from ‘assessment of learning’ towards ‘assessment for learning’ plays an important role for changing from input to output orientation of teaching and learning and support students’ critical thinking abilities. Instead of assessing how well students can reproduce knowledge imparted by the instructor (input), the focus shifts to the competencies students can apply (output).
How can instructional designers create activities that are meaningful, contextualized and connected to real-world problems? Though there is no alchemistic formula, it is important to understand that authenticity is a continuum. Gulikers, Bastiaens & Kirschner (2004) distinguish five dimensions of authentic assessment: (a) the task, (b) the physical context, (c) the social context, (d) the results, and (e) the criteria. Each dimension forms a continuum, which means that authenticity is not an all or nothing trait. Furthermore, authenticity is a subjective measure. The perception of what authenticity is may vary among individuals as a result of educational level, personal interest, or amount of professional experience (Gulikers, Bastiaens & Kirschner, 2004).
Often times, when we talk about ‘authentic assessment’ in the instructional design process, we really mean creative assessment. We are looking for techniques that are engaging, surprising, puzzling, challenging, unexpected or different. This can happen in many ways – creating a mindmap, producing a comic strip, developing an information graphic, creating a game. It does not necessarily mean to be as close as possible to the ‘real world’.
A great way to frame assessment in the disciplines are threshold concepts. The idea of threshold concepts emerged from a UK national research project into the possible characteristics of strong teaching and learning environments in the disciplines for undergraduate education. Meyer and Land (2005) characterize threshold concepts with the following qualities: transformative (significant shift in the perception of a subject), integrative (exposing the previously hidden interrelatedness of something), oftentimes bounded (demarcating academic territories), probably irreversible (unlikely to be forgotten, or unlearned only through considerable effort) and potentially troublesome (often problematic for learners, because the concept appears counter-intuitive, alien, or incoherent).
Disciplines have ‘conceptual gateways’ or ‘portals’ that lead to a previously inaccessible way of thinking in a process of liminal transition – these are ‘threshold concepts’. An example from the social sciences is that ‘you cannot make causal inferences from correlational data’. Mastering a threshold concept puts learners in a liminal state where they oscillate between old and emergent understandings – just like an ethnographic researcher who not outside, but also not quite inside the group. So one way to think about assessment is to identify the threshold concepts in the domain you are working on and coming up with creative ways to help learners traverse these portals.
Threshold concepts allow instructional designers to support assessment for learning. This type of assessment encourages students to question their preconceptions and evaluate their grasp of crucial concepts in their discipline. Within an organization, assessment for learning confronts stakeholders with their preconceived notions of organizational issues or initiatives, and fosters the shared understanding of problem scope as well as crucial components that are difficult to conceptualize.
Authentic and creative assessment is not only a goal for classroom and online learning, but also for other types of assessment that fall in the domain of instructional design such as organizational improvement or the evaluation of educational technologies and techniques.
Across domains, we feel tensions between measure and treasure. What we measure through standardized tests and metrics is not necessarily what we treasure. Vice versa, what we really care about, we oftentimes cannot operationalize. For individual learning, there is a growing dissatisfaction with standardized test scores. On the organizational level, we hear criticism about program rankings and the prevalence of the social science citation index. In educational technology research, we see the limitations of the experimental paradigm.
Educational technologies can support creative processes and offer connections to authentic contexts, just as well as they can curtail creativity and foster standardized testing routines. Once we start taking individual strengths and skills into account instead of filing students – or organizations – through standardized routines, the assessment becomes a part of the learning process, not an end in itself.
