Assesslog

To focus on how we assess, Eva’s suggested that we pick two assessment intruments we’re familiar with and analyse them in terms of a number of ‘dimensions ‘of assessment; e.g. formative – summative, standardised – authentic, etc.

The two assessment instruments I’m going to analyse are as follows:

A. An online group role-play activity (case study at go.bath.ac.uk/M3roleplay)
B. Online peer review of assignment drafts (case study at go.bath.ac.uk/ICM_peer_review)

Dimension 1: Formative – Summative
B is clearly entirely formative, while A has a blend of summative and formative elements. The formative capacity of A is mainly within the breakdown of the mark given – as you can see from the mark scheme, the student should come out of the assessment process knowing where marks were gained and lost, and the strengths and areas for development they demonstrated during the task. The students do also receive a single ‘final mark’, but they are encouraged as professionals to interpret the outcome of the assessment as a basis from which to move forward and reflect on how they might improve their skills further, as well as how to apply what they’ve learned to the situations they encounter in their workplace.

Dimension 2: Formal – Informal
Both A and B are primarily formal assessment incidences – however, the more authentic nature of A (and the fact that it is in itself an extended teaching/learning activity) means that informal self, peer and tutor assessment is likely to take place throughout.

Dimension 3: Product – Process
A is primarily about process and B is primarily about product. The mark scheme for A (linked to above) demonstrates that is it mainly process elements such as participation, facilitation and even attitude that are being assessed. The information we’ve been given about this claims that processes are rarely assessed explicitly, ‘probably because of their insubstantial, transient nature’. However, assessment activity A takes place online, and all communication is in written form – which makes the negotiation process and the involvement and attitudes of the group members more concrete and thereofre easier to assess.
We are currently amending the peer review activity (B) so that the process element is taken more into account – not in terms of how their peers have gone about writing their assignments, but in terms of how they have carried out the review. In the most recent version, the two students reviewing an assignment get to compare their reviews with each other, and then tutor reviews the review process and provides feedback to both the author of the assignment and the two reviewers. 

Dimension 4: Continuous – Terminal
I’m not sure if this dimension is relevant to ’B’ at all, as it seems to me neither continuous nor terminal…? The formal assesssment element of ‘A’ could be continuous if it took place at all three stages of the task, rather than just at the end. As mentioned above, with ‘A’ there is informal self, peer and tutor feedback taking place continuously.

Dimension 5: Standardised – ‘Authentic’
‘A’ was initally conceived to be as authentic an assessment task as we could reasonably make it, not intentionally to the learning and teaching conditions (the assessment task itself was actually seen as part of the learning/teaching activity), but to the situations that the students would be most likely to encounter in their workplace. The groupwork element of ‘A’ certainly raises challenges for its reliability, but I would hope that the validity of this form of assessment might balance that out. ‘B’ incorporates a standardised assessment task and a standard set of assessment criteria that students used to carry out the peer review. The students are given guidance on how to use the criteria, but the lack of any sort of quantitative output makes me question whether reliability is an issue for ‘B’…?

Dimension 6: Internal – External
Oh crumbs – there is so much peer and self-assessment going on here, I would say that both ‘A’ and ‘B’ range from internal to very, very internal

Dimension 7: Norm-referenced – Criterion-referenced
Both ‘A’ and ‘B’ are criterion-referenced. It should be noted that with ‘A’, there will be probably be some discussion at an exam board about the spread of marks if it does not loosely conform to a normal distribution…!

In conclusion – that was quite an interesting analysis for me to carry out. I didn’t consciously have any preconceptions about how different the two assessment instruments would be in terms of the seven dimensions, but some of the analyses did surprise me a little. I also think that this analysis will come in handy when presenting my experiences of designing and using these instruments to others through case studies and workshops – great stuff