Assesslog

For Monday’s session, Eva suggested we read Chapter 3 of Gipps’ Beyond Testing: Towards a theory of educational assessment (1994), and reflect on the ways in which assessment, evaluation & testing impact on students’ achievement & motivation.

Reading this article caused me to think more deeply about the nature of achievement. It’s often the case that the abilities that are tested (i.e. the ones that are easiest and most economical to test), become the ones that are the most taught; there is a bias against teaching those skills that are not, or cannot easily be, measured. Therefore, when a student appears to achieve well, one has to consider the scope of the test’s validity, and the constructs or skills that it is actually measuring.

Gipps highlights Heisenberg’s uncertainty principle – that one cannot measure things without affecting them – and describes the concept of construct-irrelevant variance, or ‘test score pollution’, where test scores may rise due to teaching being focused on the test items and formats themselves, rather than on the constructs or skills they are intended to measure. This is something else to consider; whether an apparent increase in achievement is due to a greater understanding of a construct or better grasp of a skill, or largely due to students being more highly skilled at taking tests.

It is difficult to imagine how the negative effects of teaching to the test can be minimised with the existence of high-stakes testing. As Gipps points out, teachers feel that they have a professional duty to give their students the best chance to pass those tests that will have a significant effect on their lives. Gipps asks whether even a positive impact such as that created with the move from ‘O’ levels to GCSE can become corrupted over the years.

Reflecting on these points helped me to see my own experiences with a fresh perspective. I gained nine ‘A’ grades at GCSE but received a ‘D’ grade in my English Literature examination. Initially I was surprised and disappointed, and felt that the test must be at fault, but over time I began to see the D grade as an absolute and valid measure of my ‘achievement’ in this area. Gipps’ description of test results as ‘a useful but fallible indicator of achievement’ is helpful. It encourages me to examine what exactly my English Literature examination was testing. One might assume that the aim of an English Literature assessment would be to assess a student’s appreciation of good literature. However, this is not an easy quality to measure. The ability to unpick a metaphor, however, is a much easier quality to measure, and I suspect that whoever wrote my English Literature paper was thinking (hoping?) that measuring someone’s ability to deconstruct metaphor was the same thing as measuring their ability to appreciate and enjoy literature – as Gipps phrases it; generalising to other measures of the same construct. Ah well – at least they picked a measure that was resistant to pre-test coaching (not that my English teacher didn’t try). I felt thoroughly prepared (i.e. coached) for the other nine exams, so I’m not suggesting that those results are any less fallible. Society, however, will still simply see me as a ‘high achiever’ in these subjects and a ‘low achiever’ in English Literature. Gipps summarises the conclusions made by Madeus (1988) - that we as educators need to lower the stakes of tests, and also to try and persuade everyone that test results are only one piece of the puzzle.

Taking the latter point - it seems clear to me that having more detailed reporting of results is crucial in order to discourage the making of simplistic assumptions about achievement – this relates to the issues of unidimensionality and universality discussed in my earlier post on Assessment Paradigms.

Looking now at the issue of motivation (which is inextricably linked to achievement), and how it can be affected by assessment/evaluation/testing, the way forward seems a little clearer, particularly with low-stakes assessment; what Crooks (1988) terms ‘classroom evaluation activities’, as I’ll go on to describe below. With high-stakes assessment we have a difficult conflict between the external motivation experienced by students who believe they can succeed and therefore obtain the rewards, and the demotivating effect on those who know they cannot.

According to Crooks (1988), classroom evaluation activities serve to emphasise the skills, knowledge and attitudes that are valued, help to structure approaches to study and consolidate learning, and affect the development of enduring learning strategies. Crucially, when classroom evaluation is used within a framework of attainable sub-goals, each with clear criteria, it affects self-perceptions of competence (self-efficacy), which is shown to be closely related to the use of deep learning strategies and the ability to persist with challenging tasks. The term ‘classroom evaluation activities’ can also be applied to the input from the e-moderator or e-tutor in the online distance learning programmes I work with. A significant part of my role is to encourage the tutors to engage more deeply with the online activities, and I always see evidence of an instant and powerful impact on the students’ motivation when their tutor provides specific, well-timed feedback on an activity.

Norm-referencing can have a negative impact on motivation; not only can it discourage collaboration and threaten peer relations; it essentially attributes success and failure to ability rather than effort.

The GCSE was intended to emphasise positive achievement (and therefore enhance motivation) by allowing students to show what they could do rather than facing them with impossible tasks. This required the use of differentiated tasks and criteria for different levels of expected achievement, which could unfortunately have a demotivating effect on those who realised that, however hard they worked for their maths GCSE, for example, they would only be able to attain a maximum B grade. However, the changes in content and teaching brought about by the move from ‘O’ level to GCSE, particularly in subjects such as MFL, resulted in a massive increase in the number of students choosing to progress to ’A’ levels.

I’m off to the gym now – but afterwards I’m going to look into classroom evaluation practices more deeply by reading Terry Crooks’ 1988 article on the impact of classroom evaluation practices and reflecting on how the key points he makes relate to my own context of online learning environments :-)