Assesslog

The third text Eva asked us to read in the first week was on the social purposes of assessment – from Education, Assessment & Society: A sociological analysis, by Patricia Broadfoot (1996). This one took me a while to read – I tend to write my questions in the margins when I don’t understand what I’m reading, and it seems that Pat Broadfoot tends to write something first and explain it later, which means I used up a lot of ink.

What Eva asked us to reflect on while reading this text was: “How does the social purpose of assessment impact on the content of assessment and on the teaching content?”

I found the scope of this text very broad – looking as it does across cultures and centuries – but I felt that it provided me with a better insight of the historical and cultural forces that affect the educational system, and a further realisation of the stregth and complexity of the ties between assessment and learning.

In answer to the question of the impact of social purpose on assessment, a key theme of the paper was the differentiation of roles in what Broadfoot calls ‘complex’ societies, and the differing social status of these roles leading to competition and therefore the need for selection. Broadfoot attributes the existence of free competition to historical periods of upheaval such as the Enlightenment, which gave rise to inidvidual rights, scientific advancement and - most importantly – meritocratic ideologies, and the industrial revolution which brought about the need for formal certification of skills, and a move towards contractual employment. Selection for such roles requires the measurement of attributes against a standard; hence the need for standardised tests. The issues of status and power were also discussed in some detail; Broadfoot raises an interesting point in that the high status of certain roles permeates into the actual assessment methods through which entry to those professions is sought.

Broadfoot argues that assessment procedures may not be reinforcing the knowledge and skills that are actually required by society and the economy, but an arbitrary representation of knowledge selected by those in positions of sufficient power to determine what kinds of intellectual activity have the highest status.

Broadfoot brings many factors into explaining the pervasiveness of the written examination in our education system; the historical and cultural reasons for its dominance and the effect this has on the curriculum.  Two key ideas Broadfoot highlights are the reliability of the written examination (important for comparability when dealing with the pressure of large numbers and the need to select), and its links with the testing of innate general ability, which countries like the UK and the US still believe in as a predictor of future achievement. Broadfoot argues that the alternate view that achievement is the result of persistence is one of the reasons for the success of the Japanese education system. I’m tempted to think that perhaps neither view is wrong and that success is largely due to an innate ability (& desire) to persist – but I can see that putting the emphasis on persistence opens up the possibility of success to all. 

The second core reading of the week I’ve just thoroughly enjoyed digesting was Chapter 2 of Elliot Eisner’s The Art of Educational Evaluation, titled ‘Educational Objectives: Help or Hindrance?‘.

Before I started reading the chapter, I read a bit of background about Eisner, his life and his work - and I’m glad I did as it helped me to get an idea of the context in which he was writing. Eisner’s educational origins in the teaching of art history give some insight into his views on educational work as a form of creative artistry, and the parallels he makes between education and art.

To me, this article was quite a radical deconstruction of the assumption that the curriculum design process has to start with the statement of educational objectives, and to use a metaphor (which I generally don’t), it rocked my foundations a bit. In bish-bash-bosh format, Eisner’s argument against starting the curriculum design process with the statement of educational objectives is as follows:

1. We can’t accurately predict educational outcomes
2. Subject matter has a significant impact on precision in stating objectives.
3. Educational objectives are often assumed to be standards for measurement when in many situations they can only be used as criteria for judgement.
4. Teachers often start with means (’what am I going to do’) and then relate them to ends (’what am I going to accomplish’), and this (according to Eisner) is logical.

At first I felt that the latter point was a pretty brave thing for Eisner to stick his neck out and say. But then I thought about it some more, admitted to myself that that’s the pattern I, and my colleagues, tend to work in, and thought that maybe it’s not because we’re all lazy, incompetent bunglers – maybe it IS logical. Whenever I’ve tried doing it the other way round – identifying an objective and then thinking up an activity that will result in that objective being met - it’s not that it’s impossible, but creatively it’s a lot harder and, to be brutally honest, the activities often aren’t as much fun  

Eisner’s argument really spoke to me, although I must admit I still see the rationality behind the argument for starting with objectives – I trained as a personal fitness trainer several years ago, and the best way to get clients in shape (and, frankly, to get clients in the first place) is not by suggesting ’lets run around and lift some weights and see what happens’. But I’m willing to concede there may be subtle differences between education and physical training (on reflection, the physical training metaphor seems a very good description of the traditional approach to setting objectives – and the theory of specificity of transfer too!).

More later…

Eva suggested we have a look at the first chapter of Gipps (1993) – Beyond Testing. As expected from an introductory chapter, this text threw out a bundle of interesting concepts to grapple with.

I was startled to be confronted with the concepts of construct validity and factor analysis, a year after finishing my Research Methods assignment and feeling relieved that I’d never have to think about them again (yes, I was delusional). The paradox presented by the assumptions of universality and unidimensionality was probably the muddiest point of the chapter for me; however, I think the points that was being made were: 1) as many attributes (e.g. reading) are multidimensional, attempting to assess only one dimension of an attribute (e.g. comprehension) is illogical, and 2) if multidimensional attributes are being tested, two identical scores (for example) will not tell the same story. These points fit with the concluding paragraphs of the chapter, where it is implied that (regardless of the purpose of the assessment, I think), both the assessment and the presentation of the outcomes should ideally be in a format that gives a fuller picture of exactly what competences pupils and students have – notwithstanding the fact that a student’s learning will not necessarily fit neatly into an ordered set of performance descriptors. Gipps uses the National Curriculum as an example, which I found rather amusing – I remember playing at a friend’s house – it must have been around 1991 - and listening to my friend trying to explain to her mother what her recent maths scores meant. Her mother just wanted to know a) a percentage and b) where her daughter was in relation to the rest of the class. To be honest, that was all we wanted to know at the time as well! Recalling this made me realise just how pervasive such paradigms can be.

This first chapter focused mainly on the different purposes of assessment – e.g. monitoring, accountability, supporting learning, and the fitness for purpose of different styles of assessment. It makes sense that tests and exams used for monitoring and accountability at a class, school or national level will need to be quick, manageable and reliable. If assessment is to be used for the purpose of certification, there will need to be more of a balance struck between efficiency and depth/detail. However, assessment used on an individual level, for the support of learning, is a rather different kettle of fish – it needs to be detailed in quite a granular way so that individual strengths and weaknesses can be identified and ‘executable advice’ (Glaser 1963) can be given, and it needs to measure competence (best performance) rather than average performance. There are evidently problems with reliability in this kind of developmental assessment, and Gipps cites the work of Stiggins (1992) in suggesting that the differences between the types of assessment are so fundamental that attempts should never be made to blend the two.

Gipps highlights quite frequently that, as different assessment practices (via their effect on teaching) encourage different styles of learning, it’s vital to consider the kind of learning that we wish our students to achieve when choosing or designing a method of assessment. Gipps cites the work of Goldstein (1992) in reminding us that assessment also impacts directly on students’ learning to some extent – students never pass through the assessment process totally unscathed – it will always result in some kind of learning. Gipps uses the UK SATS as a shining example of authentic assessment (sometimes called ‘performance assessment’) that models the learning activities taking place, and therefore doesn’t lead to so much distortion of instruction. My year group was the first year to sit the Year 9 SATS. I remember feeling particularly bereft at the lack of ‘coaching’ that took place beforehand – we were so used to being given past papers for homework, practicing in exam conditions, etc – we really felt the difference and it wasn’t comfortable for us at all to not have what we saw as formal preparation for the assessment process itself. Many of my peers did rather badly, simply because they weren’t used to being asked to think rather than to recall - they assumed the questions were asking them to regurgitate some sort of prior knowledge that they didn’t have, so they left it blank and moved on.

I’d be interested to hear about anyone else’s experiences of the National Curriculum, and SATS, and how they resonate with the concepts that Gipps presents…!

I finally handed in my blogging assignment last week – phew! Those of you who are interested can read the whole thing here on Googledocs, and the accompanying blog at www.metablog.edublogs.org.

So…less than a week later and here I am embarking on another unit in the MA programme – and this one’s on Assessment. I’m chuffed to bits that this time I get to share the journey with six other learners; Bill, Nina, Tim, Naeema, Nicola, and Debbie – we had some fun conversations this evening during our session and I’m already looking forward to next week.

In our first session we did a lot of thinking about the purpose of assessment, and some key issues related to assessment.

In my own professional context, three key issues important to me are:

• Relevance and authenticity of assessment (I work with professional distance learning students)
• Development of learning communities to facilite informal peer assessment & feedback
• Engaging students with assessment criteria (for example, through formal peer assessment)

It’d be interesting to share our top three assessment issues… readers - it would be great to hear about yours

What I found most interesting about this session was that we looked at examples of assessment outside formal education – I found that this allowed me to look beneath my own past experiences and assumptions to get a clearer picture of what assessment actually is. The example of mothers engaging with their children in play was great - we could see a constant stream of assessment, feedback and guided participation taking place naturally and instinctively.

Over the next few days I’ll be posting my responses to some readings, plus having a think about a couple of assessment instruments in terms of dimensions of assessment – formative/summative, formal/informal, etc.