Examinations: After the Gold Rush

In my previous two blogs I looked at some of the serious problems which exist in the marking of subjects like History and English at GCSE and A Level, and at potential changes which could be made to improve the reliability of examinations. However, I also noted that such modifications might not resolve all of the problems identified.
 
In this final blog, I want to explore a more radical solution to the search for the “gold standard” of examinations: it’s abandonment. Indeed, to take a gold rush analogy, it was seldom the gold hunters who profited much from the great gold rushes in America. In fact the gold hunters gave way to huge corporate interests and long term destruction was the result (though the companies certainly did well). Instead it was those who supplied the tools, cooked the food, cleaned the cabins, and provided the clothes who really made the profits (most notably of cause one Levi Strauss). In short, those people who recognised that the opportunities lie in the everyday, not the elusive. So what would this look like?
 
First, I want to suggest that we need to reconsider the purpose of summative assessment in schools. Up to now, examination has been seen only in terms of measuring the standardised “outcomes” (and thereby potential) of students and schools. However, I would suggest that well designed assessment should in fact be supporting the development of rich curricula, improving teachers’ engagement with their subjects, and promoting deep curricular engagement among students. This in turn would impact on students’ knowledge and understanding, and thereby implicitly their outcomes.
 
Second, and in order to achieve the above. I think the creation of assessments need to be devolved to the level of schools, or groups of schools working together. This is not the same as saying all work should be coursework, just that the assessments should be designed and set in smaller, local groupings. In such as system, students learning might not be so easily comparable nationally (though this clearly isn’t working well in some subjects anyway), but the improved quality of teaching might well mean better outcomes in real terms, regardless of the grading systems used.
 
Why are such changes needed?
To understand the power a locally led examination system might have, one must first focus on the problems inherent in assessing a subject, like History or English, where there is no definitive agreement on content at a national level. I have outlined a selection of these below:

1. The existence of a national specification tends to result in curricular narrowing. Most GCSE Germany exam specifications used to begin in 1919 (and some controversially ended in 1939!) and therefore few schools bothered to focus on much before this date, despite the fact some knowledge of Germany from 1870 onwards is exceptionally helpful to appreciating the difficulties faced by the Weimar government especially. Devolving responsibility to local groups would allow more careful consideration of the necessary breadth of content to be covered.

• National exam specifications are also very vague. For instance, one bullet point of a history specification (representing a week’s worth of work) says that students need to study:
  
  “Increasing conflict on the Plains: the Fort Laramie Treaty (1851) and the failure of the policy of concentration; the Indian Wars (1862–1867): reasons for and consequences of the Wars; Sand Creek Massacre; Fetterman's Trap.”
  
  It is not made clear what the focus on “increasing conflict” should be (The nature of? The causes of? The results of?). Equally it is unclear what exactly about the Fort Laramie Treaty students should be taught (The key signatories? The major terms? The disputed terms?). And then of course there is the issue of the “Indian Wars 1862-7”. Which ones? There are ten major Indian Wars in this period. Teachers are left guessing what to include and, critically, second-guessing what the exam board might ask about. Any one of these specification items might reveal a nasty surprise: “Explain the causes of the Goshute War of 1863 [12]” or “Name the date on which the Fort Laramie Treaty was signed [1]”. Teachers are in a situation where they either madly cram in as much content as possible (a common approach), or they start second-guessing which content it might be safe to leave out. Indeed, in the last twelve months I have come across at least two schools who were trying to predict which specification items they could ignore because they “wouldn’t come up” in the summer. This then becomes another form of curricular narrowing.
  
• The problems inherent in exam specifications (and the ways they are often misunderstood) means that textbooks have become a defacto source of content. This is mainly because they plot a route through this course content. Teachers then feel duty bound to “buy in” to the branded book in the hope that they will be given the “right” content to cover. Of course, this is not mandatory, but the textbooks are often the same ones the markers then read in preparation for their marking and so the choice of book is reinforced. This was especially problematic when key book authors also set and marked the papers resulting in some questions on topics quite niche to specific textbooks. Of course, the use of the textbook also comes up short a lot of the time. Despite the trend for branding books, examiners will set questions from the specification. Sometimes teachers rely on textbooks so much that they fail to consider their teaching in relation to the specification at all and therefore spend little time on aspects of the specification which the textbook glosses over. Either way, teachers (often for the best reasons) find themselves outsourcing their curricular thinking to others.
  
• Far too often, teachers rely on a general knowledge of the historic versions of the specification in determining what to teach. For example, in the past, the “Indian Wars” has meant the Colorado War and Red Cloud’s War. Sometimes this backfires (especially evident in the new Medicine Through Time specifications which only cover Public Health) and teachers waste time covering things in detail which could have been done briefly. More often though, this knowledge privileges those who have an insight into the historic versions of the specification, not least because their answers also shape what is seen as acceptable by the board. However, the reliance on historic specification knowledge in teaching has a more insidious impact, fossilising exam specifications in outdated historiography. A good example here is the inclusion of the “Fetterman’s Trap” in the above specification. This is arguably a tangential issue in the development of the Indian Wars, but was popularised as an event by Dee Brown’s (1970) “Bury my heart at Wounded Knee”, and therefore continues to feature. This in turn impacts on the power and relevance of the history curricula planned for these GCSE students.  

• Assessment objectives offer very little help in determining how to teach and almost none on what to teach (something I have written about before). For history the four objectives are given below. This is a picture painted with such broad brush strokes that I suspect even Jackson Pollock would be embarrassed to submit it as a finished piece.

Curricular empowerment
If schools worked together to build and design their own specifications and assessments (graded or matriculation style), I believe it would overcome many of the issues outlined above and pave the way for teachers to be better engaged in professionally enriching work at Key Stage 4 and 5.

1. Without the focus on vague specifications, teachers would be free to build meaningful courses based in shared knowledge. This would be incredibly enriching and allow some of the most exciting work at Key Stage 3 to be replicated post-14. I suspect this would also do wonders for recruitment and retention of teachers as I am not sure who really finds joy in delivering pre-set content to a frankly baffling array of pre-determined assessment objectives. As one of my excellent colleagues, commented: “teachers need to keep sight of why they became teachers in the first place – it wasn’t to meet performance management targets over GCSE grades or become desperately familiar with a set of assessment objectives.”
2. Teachers would be able to set assessments which sampled appropriately from the domain they had taught. They would also recognise the need to not reduce their teaching to just the sample because they would have been involved in choosing the appropriate hinterland for their course, and therefore presumably see its value. They would also be able to set more rigorous, interesting and potentially more valid assessments as they would serve a specific purpose in an agreed curriculum, rather than a broad purpose in a genericised national curriculum.
3. Teachers would have the freedom to update and review their curricula in light of new and recent research. This would open the door to professional development work, especially if specification renewal was made part and parcel of such a system. Again, I am sure this would do wonders for teacher retention and could tap into brilliant projects such as the HA’s Teacher Fellowship Programme.
4. Teachers would be able to examine their students more accurately due to their increased knowledge of what exactly had been taught. It would still be possible for schools to make claims about the abilities of their students and therefore to provide some information about the outcomes and potential of such students.

Monitoring
Of course, this is not an exhaustive plan and I am sure there are many gaps which still need addressing. One major area for thought is the oversight of such a system. Again, this feels like less of a big leap now than it might have been before the breaking up of the LEAs and the creation of MATs.

1. Ofqual could set broad subject parameters which local exam groups had to consider when designing their specifications. A good example of this can be seen in the Queensland System, where 75% of all examination is set at a local group level (QSA, 2010).
2. Carefully designed, but very broad, exam specifications could be set out in a similar manner to some parts those offered by OCR B for history. For example, one bullet point of the America specification says “How and why the USA expanded, from 1789 to 1838.” Because no specific causes are listed, none can become a “trip up” question e.g. “Explain how the Northwest Ordnance contributed to American expansion” could not be asked, but “Why did America grow 1789-1838?” could. This means schools are actually free to teach the nature and causes of expansion as they see fit. Similarly, the “History Around Us” unit which sets 14 broad criteria around which schools have to design an appropriate course. Although there are still many issues with the way this specification is assessed, parts of the specification do offer a very good model.
3. Ofsted are already moving towards judging the quality of curricular provision rather than solely examination results. It would be a small step to expand this to look at curricular design in Key Stage 4. The shift away from high stakes grades in a matriculation system might also lessen the pressure on schools to “cheat” and boost grades.
4. The money saved from national exam entry costs could be spent on appropriate verification and moderation. This might operate in a similar way to universities, where other institutions or groups give oversight of assessments and course design. Equally, Ofqual might appoint key persons to such a role. It seems easier for one person to get acquainted with he approaches and assessments of a few dozen schools than to try and get consistent understanding amongst thousands of teachers of a single specification.
5. There is no reason that formal examinations could not be used, and external invigilators appointed for oversight. In terms of marking, this could be shared between subject teachers in a local examination group and checked for rank ordering at a face to face meeting, or marked using CJ. When we used to mark coursework, the key measure at the end was to ensure the rank ordering was correct. This was seldom a major problem and was often a good way to understand how to improve teaching for the following year. This would be good professional learning.
6. There might be some argument that certain subjects, or component parts of subjects, could continue to be assessed nationally where there was enough agreement on content. For example, Maths might continue to work as a national examination, as might some elements of the science curriculum. We might also introduce a national spelling and grammar test for 16 year olds. This is similar to the approach taken in Sweden.
7. Businesses and universities seeking to have more detail on their applicants could set their own entry tests/interviews based on the qualities they were looking for. References are already in place for UCAS, so more use could also be made of these.

​

There are in fact already large-scale instances of similar systems in operation. For example, the Swedish system places grading in the hands of teachers at a school level and national exams, though on offer in Swedish, Maths, and English, are generally seen as unnecessary (Wikström, 2006). It is fair to say however that this system is also based on a more generic competencies view of curriculum and would therefore need adapting to live up to the suggestions outlined above. A good UK example was the OCR GCSE History Pilot which saw schools setting their own topic areas for assessment through internally set exams. These were still beholden to broad mark schemes (which could be refined) and therefore the grading was still done nationally. Having taught this specification in the past I can certainly say that I found it incredibly professionally enriching. Indeed, the only unit where there were ever problems was the nationally examined one on the power of kings. This model could be certainly be refined and adapted with the right will and appropriate support. 

After the gold rush
Over the course of these three blogs, I hope I have managed to outline why I think we need to move beyond the notion that national examinations are somehow a universal “gold standard” of assessment. In my view, fundamental rethinking needs to happen about the system of examination. This might allow it to improve educational outcomes for children in a deep curricular way, rather than superficially through a drive for elusive “gold standard” examination grades. I believe that moving away from national examination may hold benefits not only for students, but also by empowering the teaching profession more broadly. I am not sure if this blog has done much to get towards solutions, but I do hope it opens up some interesting discussions for what life might look like after the gold rush.
 

References
 
Queensland Studies Authority (2010) School-Based Assessment: The Queensland System [online] Queensland Studies Authority. available from <https://www.qcaa.qld.edu.au/downloads/approach2/school-based_assess_qld_sys.pdf>
Wikström, C. (2006) ‘Education and Assessment in Sweden’. Assessment in Education: Principles, Policy & Practice 13 (1), 113–128