In my previous two blogs I looked at some of the serious problems which exist in the marking of subjects like History and English at GCSE and A Level, and at potential changes which could be made to improve the reliability of examinations. However, I also noted that such modifications might not resolve all of the problems identified.
 
In this final blog, I want to explore a more radical solution to the search for the “gold standard” of examinations: it’s abandonment. Indeed, to take a gold rush analogy, it was seldom the gold hunters who profited much from the great gold rushes in America. In fact the gold hunters gave way to huge corporate interests and long term destruction was the result (though the companies certainly did well). Instead it was those who supplied the tools, cooked the food, cleaned the cabins, and provided the clothes who really made the profits (most notably of cause one Levi Strauss). In short, those people who recognised that the opportunities lie in the everyday, not the elusive. So what would this look like?
 
First, I want to suggest that we need to reconsider the purpose of summative assessment in schools. Up to now, examination has been seen only in terms of measuring the standardised “outcomes” (and thereby potential) of students and schools. However, I would suggest that well designed assessment should in fact be supporting the development of rich curricula, improving teachers’ engagement with their subjects, and promoting deep curricular engagement among students. This in turn would impact on students’ knowledge and understanding, and thereby implicitly their outcomes.
 
Second, and in order to achieve the above. I think the creation of assessments need to be devolved to the level of schools, or groups of schools working together. This is not the same as saying all work should be coursework, just that the assessments should be designed and set in smaller, local groupings. In such as system, students learning might not be so easily comparable nationally (though this clearly isn’t working well in some subjects anyway), but the improved quality of teaching might well mean better outcomes in real terms, regardless of the grading systems used.
 
Why are such changes needed?
To understand the power a locally led examination system might have, one must first focus on the problems inherent in assessing a subject, like History or English, where there is no definitive agreement on content at a national level. I have outlined a selection of these below:

Note: my final blog offering suggestions for real examination reform is now live here.

In my previous blog I looked at the ways in which the marking of examinations in England is particularly problematic in subjects like History and English. For a full review of this, you may like to read Ofqual’s blog and report on the subject. Today I want to deal with the question of what action the educational establishment might take.

GCSE and A-Level examinations are still being held up vital to the education system. The impetus has been to seek to cement their place as the “gold standard” both nationally and internationally. If we want this to be true in reality, I wonder if we need a more fundamental rethink of what examinations (especially GCSE examinations) are for and therefore what they might be in an educational landscape which is vastly different from that of the late 1980s when GCSEs were first introduced.

What are GCSE examinations for?
The seemingly simple question of the purpose of GCSE examinations is actually very complex indeed. But of course, as with all assessment, validity is heavily connected to the inferences one wants to draw from an exam (Wiliam, 2014). The range of inferences which are suggested as valid from a set of GCSE examinations are extremely diverse:

Note: there are two blogs which follow this one which offer some solutions to the problems outlined here.

A few days ago, Ofqual published an interesting blog looking at the state of the examinations system. This was based on an earlier report exploring the reliability of marking in reformed qualifications. Tucked away at the end of this blog was the startling claim that in History and English, the probability of markers agreeing with their principal examiner on a final grade was only just over 55%.

The research conducted by Ofqual investigated the accuracy marking by looking at over 16 million "marking instances" at GCSE, AS and A Level. The researchers looked at the extent to which markers’ marks deviated from seeded examples assessed by principal and senior examiners. The mark given by the senior examiner on an item was termed the “definitive mark.” The accuracy of the other markers was established by comparing their marks to this “definitive mark.” For instance, it was found that the probability that markers in maths would agree with the “definitive mark” of the senior examiners was around 94% on average. Pretty good. They also went on to calculate the extent to which markers were likely to agree with the “definitive grade” awarded by the principal examiners (by calculation) based on a full question set. Again, this was discussed in terms of the probability of agreement. This was also high for Maths. However, as noted, in History and English, the levels of agreement on grades fell below 60%.

When Michael Gove set about his reforms of the exam system in 2011, there was a drive to make both GCSE and A Level comparable with the “the world’s most rigorous”. Much was made of the processes for making the system of GCSE and A Level examination more demanding to inspire more confidence from the business and university sectors which seemed to have lost faith in them. Out went coursework and in came longer and more content heavy exams. There was a sense of returning GCSE and A Level examinations to their status as the "gold standard" of assessment. The research conducted by Ofqual seems suggest that examinations are a long way from such a standard. Indeed, it raises the question of whether or not national examinations have never really been the gold standard of assessment they have been purported to be. Have we been living in a gilded age of national examinations? The answer is complex.

Before I launch into this, I should also note that I understand the process of examining is a difficult one and that I have no doubt those involved in the examinations system have the best interests of students at heart. I also don’t want to undermine the efforts of those students who have worked hard for such exams. That said, there were some fairly significant findings in the Ofqual research which need further thought.