Following on from my previous post on the impact of assessment, evaluation and testing on achievement and motivation, I’m going to narrow my focus on the impact of classroom evaluation practices in particular, drawing on Terry Crooks’ 1988 article – The Impact of Classroom Evaluation on Students – and thinking about how his recommendations for educational practice relate to my own context of online distance learning.
As summarised in my previous post, there is significant potential for classroom evaluation activities to impact positively on students’ achievement and motivation. Crooks emphasises a number of points that should be considered and acted on in order to maximise these positive impacts:
1. Encouraging deep learning strategies
Crooks points out that classroom evaluation tasks should be based on the demonstration of understanding, through the application of learning to new problems, and other higher order thinking skills.
In my context, working with masters-level students, it seems clear that formative evaluation of students’ ability to simply recall information is not something that we should be focusing on. I’ve always wondered about how useful the self-test quizzes are to our students. I feel that we should be concentrating our efforts on activities that require students to use factual knowledge to solve a problem or carry out a process – for example, the Module 3 role-play activity, the group discussions in Modules 4 and 6 and the case study exam practice in Module 5. Ideally the reflective discussions in Modules 1 and 2 (soon to be expanded into the other modules), could be varied over time to bring in a problem-solving flavour, perhaps by alternating introspective, reflective questions with problems or questions to be answered in a collaborative way.
2. Assisting learning through evaluation
Crooks reminds us of the risks of normative grading (see my previous post) and advocates minimal use of summative grading with classroom evaluation activities, and an increased focus on the identification of strengths and weaknesses through useful formative feedback. This is something that we are working hard to address with our online DL programmes; where the original model simply did not incorporate formative evaluation; tutors were employed to mark assignments and examination scripts. It’s simple enough for a learning designer such as myself to incorporate formative evaluation activities (discussions, workshops, etc) into the modules, but in order to have the motivation and confidence to self- and peer-assess, a significant degree of evaluation from the tutor is required. It’s not enough for us simply to offer to pay the (part-time) tutors for this extra work; they have to actually have the spare time to do it, which is where optimism can often be our downfall! Let’s say it’s a process that is ongoing, and progress is definitely being made; more quickly in some areas than others.
3. Giving effective feedback
Crooks recommends that feedback should have an emphasis on personal progress – this is difficult when the majority of our ICM tutors only mark assignments for one unit, but is one of the many reasons behind getting them more involved in the provision of formative feedback. It’s nice to see it happening in Module 1, where the tutor is contracted to provide detailed feedback on a draft assignment submission. Crooks also picks up on the importance of timing – that feedback should be provided as soon as possible after the event, and, where appropriate, so that the student has an opportunity to correct any deficiencies that have been highlighted. Crooks’ third recommendation is that feedback should be as specific as possible – i.e. saying exactly what has been done well and what could have been done better, rather than generalising across the entire piece of work.
Phil Race‘s December 06 presentation on giving fast and effective feedback is full of guidelines and great ideas for making feedback more effective. One idea I’d like to try out is the provision of a formal pathway for students to respond to the feedback they receive on their assignments; an online text submission activity where they can say how they feel about the feedback generally, ask for clarification on anything they don’t understand, and tell us (okay, by ‘us’ I mean tutors) what to stop/start/continue doing when we give feedback. Slide 3 of the presentation has some great points for making sure the timing of feedback is optimised. I’ve always liked the idea of sending out an overall feedback sheet based on common errors and difficulties the day after students submit their assignments, and then following up with personalised comments afterwards, but I haven’t as yet been able to persuade my colleagues of the benefits…
4. Maximising the benefits of co-operation
Co-operative activities can facilitate learning and motivation, and help to develop interpersonal skills and relationships between students. On an international distance learning course such as ICM, this is ultra-important as it is so much easier for students to become isolated. Crooks recommends that co-operation is particularly appropriate for complex tasks where the different perspectives and skills of the group members can complement each other – such as the group role-play task in Module 3 (different skills) and the various group discussions (different perspectives). As I mentioned previously, I think students could benefit more from these different perspectives if more of the group discussions had a problem-solving flavour than a merely reflective flavour. The reflective element is important and I don’t doubt that students find it very interesting at first to hear about the way things are done in different countries and industry sectors, but I think that using these different perspectives to collaboratively solve problems or even to answer questions, as in Modules 4 and 6, would utilise more higher-order thinking and prevent ‘sharing fatigue’.
5. Setting Standards
Crooks concludes that student motivation is highest when evaluation standards are high but attainable. However, these are subjective qualities and therefore some differentiation may be required. As ICM is a masters-level programme, the standards for summative assessment are fairly inflexible, but we can support students who may struggle to reach these standards by setting attainable intermediate targets, and assist all students by providing detailed and specific criteria for all tasks. This prevents misdirection of effort and should decrease the anxiety associated with evaluation. A personal tutoring system would allow individual students’ progress to be monitored more closely across the programme, enabling the setting of personalised goals.
6. Frequency of evaluation
It’s fairly obvious that students will benefit from having regular opportunities to practise and use the skills and knowledge required to achieve the learning outcomes of the programme, and from receving feedback on their performance. These opportunities encourage both active learning and the consolidation of learning. With part-time distance learners, one has to strike a balance between providing enough opportunity for practice, and ensuring those who simply don’t have time to take part in all the available activities are not fatally disadvantaged. I suppose the key is to ensure that the few activites that are offered have maximal benefit in terms of student learning, and are well-supported with tutor input and feedback.
7. Selection of evaluation tasks
…which follows on nicely from the point made above. Crooks advises that the nature and format of evaluation tasks should suit the goals that are being assessed. He also suggests offering a variety of tasks, potentially even giving the students a choice of tasks to complete, as this stimlates and takes advantage of intrinsic motivation.
8. What to evaluate?
To conclude his review, Crooks emphasises again that as educators we should be articulating and evaluating the skills, knowledge and attitudes that we perceive to be the most important. Even if these are hard to evaluate (e.g. reflective thinking springs to mind), we must find ways to assess them.