No smoke without fire: norm referencing and the crisis in uk examinations

I have to say that writing about examinations does not rank amongst my favourite pass times, yet as the GCSE fiasco has emerged, I have found myself constantly asking, who is really surprised? Every teacher surely has felt the pain of results day when you have no idea if your results are your doing, their doing or the doing of an examinations committee…. Accountability? You must be joking. I sat and longingly read the details of the Queensland examinations system which puts schools at the centre…I took the quote from the opening page of the document:“It cannot be over-emphasised that the mode of assessment dictates the nature of the educational experience and the quality of the relationship between teacher and pupils. Assessment is not something separate ??? a tool ??? by which education may be evaluated; it acts upon the educational system so as to shape it in accordance with what the assessment demands. You cannot have, at one and the same time, education for personal growth and a totally impersonal system of assessment. Assessment should be a bond between teachers and taught, not something which threatens and antagonises.
To humanise assessment, then, we have to make of schooling a more co- operative enterprise between teachers and pupils, and an opportunity to develop the whole range of human competencies, leading up to informative profiles. This should be the pattern of things for the immediate future; it is the way to shed the dreary, and often unjust, grading techniques of traditional education.Hemming (1980, p. 113???14)”

…and then I wrote this:

No smoke without fire ??? the inferno behind the GCSE English scandal.

How much do you really know about how exam grades are awarded? I would hazard that the recent GCSE English fiascohas demonstrated that, many teachers, the majority of students, and seemingly none of the British media actually have a firm grasp on the arcane science that happens in exam board offices every July! The worrying thing is that the unfairness shown in the English GCSE example happens every single year on a smaller scale to tens of thousands of hard working students.

When GCSEs and new A Levels were introduced they were billed as ???criterion referenced??? examinations systems. That is to say, there are a set of criteria which answers are judged against; meet the criteria and get the grade. If my history students reach the top band criterion on the mark scheme for example, they will achieve an A grade by default as they have shown A grade performance. All of this sounds much fairer than the old ???norm referenced??? system where only a set proportion of students could get the grades each year, regardless of how well people did.

Of course, the fundamental point is that this is NOT how modern examinations actually work! Let???s take an example. In a recent A2 History exam, students demonstrating a set of skills would get 80% of the marks and receive an A grade. The next descriptor required 70% of the marks and resulted in a B, 60% gave a C and so on. On results day the grades came back and in many cases I was over the moon, 20 of my 41 students had achieved 80% of the marks available and therefore shown that they had met the criteria for an A grade. Yet only 15 of those students were awarded an A. Equally, of the 11 students who gained 70% of the marks and matched the criteria for a B, over half were awarded a C instead. Why?

This is the point where most people lose interest, but actually it contains the fundamental contradiction and unfairness in our examinations system. The examinations watchdog Ofqualworks on a system called “comparable outcomes”, a theory that states that every year group of students should perform roughly the same. Exam boards are required to explain any deviation from this norm. Of course the explanation could never be that these students did better, or heaven forbid, were taught better. As a result, after papers are marked, the exam boards go through a ???quality control??? process called Uniform Mark Scaling (UMS) which recreates the beautiful normal distribution of results craved by the ???comparable outcomes??? brigade.

In practice, if too many students gain B grades, the paper willbe normalised so that, despite meeting the criteria for a B,some of these are given a C grade instead; some of the Cs will become Ds and so on. This year, students needed to get 88% of the raw marks on my History paper to gain an A grade, despite the fact that everyone with 80% plus was meeting the criteria for an A. By comparison, an alternative exam board demanded only 82% for students to get an A grade!

The issue raises itself most commonly in Controlled Assessments however, where the situation has got rapidly out of control. When at an exam board session this year, I raised my concerns that students who were meeting the criteria for B grades were actually getting C grades thanks to UMS. The solution suggested by the AQA employee was that I simply give my students a few more marks, so that when they were moved down, they would still get the appropriate grade. I was so appalled that I ended up writing a letter of complaint, to which incidentally I got no satisfactory response.

Controlled Assessment has therefore become a nuclear arms race of marking, where nothing but the very top marks will result in a decent grade, but one in which principles have to be disposed of if your students are to achieve the marks they deserve. I have sat in moderation meetings with other subjects where teachers have honestly suggested bumping up all marks by 3 or 4 to ???be on the safe side???. I have not yet opted to dispose of my belief in fair and accurate marking, so sadly each year my students are penalised for the inflated marking of others. The worst part is, that those teachers who don???t get to go to exam board courses because their schools cannot afford to send them, are often the worst affected. It???s not what you know about your subject you might say, but what you know about playing the exams game!

The method of awarding grades is a travesty for a number of reasons. Firstly it means that students who are actually demonstrating all the skills of top class historians are not being rewarded with the right grades, making the ???criterion referenced??? nature of the exams a cruel deception. Secondly, students are having their college and university applications jeopardised despite potentially demonstrating greater abilitiesthan students in previous years or who have sat different papers. Finally it calls into question the purposes of improvingteaching at all. If there will always be ???comparable outcomes??? then surely the only way to improve is at the expense of others. There is no future where everyone improves as this would upset the normal distributions of the statisticians and outrage those ceaselessly whining Tories who claim that exams are getting easier each year. Ofqual???s take on all this is very simple, they state that they ??????don’t want to see year-on-year increases???as this has a real impact on public confidence. I think it is time we wrestled control of the examinations from the politicians and statisticians and started to think carefully about whom these examinations are supposed to be for. Are they about petty political point scoring and appeasing the apparent legions of Daily Mail reading masses who look back with rose tinted spectacles and declare that exams are easier than they were back in t??? day. Orare they about rewarding students for their achievement, giving them a fair assessment of their abilities and allowing them to have the best chances in life?

So what is the solution? It would appear that norm referencing is fundamentally unfair and criterion referencing is too complex to administer nationally. We need to give control of examinations back to those best qualified to know and assess the pupils, the teachers. Why should teachers spend their time trying to second guess examiners and perverting their courses to match ill-devised mark schemes? The fetishisation of national standards has distracted us from the purpose of education which should be a learning and humanising experience, rather than a dash for grades (and then stopping those grades from actually counting).

A lovely idea you might think, but school level course setting and examination can, and does work in many countries.Giving control back to the profession will reap rewards in terms of course quality and ultimately student outcomes in the broadest sense. But for this to happen we need a radical change in the way government deals with teachers as professionals as well as increased funding to put teacher training on the same footing as in Germany or Denmark, as a proper profession. Sadly with the cuts to teacher training budgets and increasingly centralised directives, it seems that we are currently moving further away from this ideal. My grandma always used to say  "you'll only get out what you are willing to put in." I suppose then when you put fuck all in...