The Next Revolution: Progress, Attainment and Whole School Assessment Systems

So yesterday I delivered a session about progression in History in a post-levels world at the Leeds Learning Network History Conference. The most valuable part of the day for me, other than the excellent sessions, was the chance to speak to other heads of department about how their schools were approaching the issue of assessment and reporting in Key Stage 3. The major worry across the board was that we end up replacing the Levels system with something essentially identical. In the worst cases, schools seemed set on keeping Levels and not updating assessment and reporting arrangements at all. The main reason for this seems to come down to the age old issue (or is that actually a recent issue) of accountability. 

This got me to thinking. Is there any point in replacing our progression systems if we end up keeping a debunked system of assessment and reporting? Now I completely accept that schools are (in our current culture) going to have to show evidence of pupil progress, as it forms a major part of the Ofsted framework. However, I think there may be some ways we can make something which both satisfies the need for data reporting and allows us to develop and use our meaningful models of progression which we have been crafting over the last few months. Once again, I would like to thank Helen Snelson at the Mount and Michael Fordham at Cambridge for their inspiration on these issues! What is crucial for me, as for many others, is that we don't let our progression revolution die a death at the hands of data systems wedded to an outmoded way of thinking.

Before we get into the main  discussion, here are a number of key issues we should bear in mind (these are key things to communicate to SMT I think):

1. The reason levels were scrapped is because they were meaningless to parents and students alike - they only offered generic advice and didn't help parents to really understand how their children were doing. To quote Michael Gove, "We believe this system is complicated and difficult to understand, especially for parents. It...encourages teachers to focus on a pupil’s current level, rather than consider more broadly what the pupil can actually do. Prescribing a single detailed approach to assessment does not fit with the curriculum freedoms we are giving schools.” (DfE, 2013)
   
2. If we replace a system of levels without rethinking how we assess then we are likely to end up back where we started ie. having a linear progression model which doesn't help kids understand how to improve.
   
3. Parents most want to know where their kids are now and whether they are getting on alright. The less complicated this is, the better! A comment left by a member of a school SMT on a preivous post noted that in his survey of what parents wanted to know, three strands were highlighted: "Is my child working at the right sort of level for their age? Is my child making the sort of progress you expect? Can there be consistency between subjects so that we can understand the reports?"

Definitions 

We also need to give some serious consideration to what it is we want our assessment and reporting systems in school to measure. To begin with, I think we need to clear up some definitions which have become somewhat blurred in Ofsted speak over the years.

Attainment - a measure of understanding at a particular point or in a particular assessment (for example an end of unit test, an end of lesson assessment, an end of year exam etc.). NC Levels were always intended as best-fit end of Key Stage measures of attainment. Attainment is effectively a summative mark (ie. a grade A*-E, Fail, Pass, Merit, Distinction etc.)

Progress - a moving measure over time. This is a holistic measure which should DESCRIBE how well a child's abilities, knowledge, understanding etc. have developed. Therefore progress cannot be pinpointed with a grade, it must be described as a process ie. is the progress slow, good, rapid etc? Of course the oversimplification with KS3 Levels has come because progress has been defined as movement between two data points, regardless of the fact that these assessments are targeting different topics, concepts, skills etc. This is an erroneous use of KS3 Levels to describe progress, a task for which they were never designed.

Let's take an example: Jane Smith studies the reasons why William won at Hastings in term 1, the significance of the Reformation in term 2 and interpretations of the Civil War in term 3. In each term she is assessed and achieves a NC Level 5. By the conventional logic, she has made no progress. This is clearly absurd - firstly, there is no parity between what she was assessed on (conceptually), and secondly she has understood each of these topics well and has deepened her historical understanding. Clearly Jane has made progress here, so why would we report that she hasn't? The only way we could make such a claim would be if we had assessed her on all 3 units from the beginning - then we would expect her to develop as she learned more content. Evidently we cannot assess students by assessing them on the whole Key Stage at every assessment point, therefore any kind of system which tries to conflate attainment and progress is doomed from the start.

Progression Model - I have discussed this at some length - a progression model is the system which underpins how we help students to get better at our subject. As far as History is concerned, NC Levels have never really provided an adequate model for this and so we have been left with some hoops to jump through.

So here is the problem. NC Levels, although never intended for this purpose, became the panacea of assessment and reporting in schools. The problem was that they were not suited for reporting anything except attainment at the end of a Key Stage (I may go into this in more depth later)

Redefining Reporting

So how can we move ourselves on, and convince our schools to reject the systems they have for reporting Key Stage 3 attainment/progress? Here is one solution which I believe may be a starting point. In this case I am assuming that the school reports to parents three times a year, which seems fairly standard across the board.

Reporting Attainment
Report attainment in a unified form. Provide parents with a student's most recent assessment grade. This data should be based directly on an actual controlled assessment (or an average of a number if you do a lot) which has a specific mark scheme and is tackling a specific issue or question. For example, the mark a student got on their "Why did William win the Battle of Hastings?" assessment. 

How attainment is reported is up to schools, but it should not imply that it is a ladder of progress. For example schools might report choose to report attainment as Fail, Pass, Merit, Distinction (as Michael Fordham suggests). Equally schools might adopt a simple percentage marking system for assessments. Or of course they may adopt a system like GCSE and have grades, or points, or whatever. The point is, whatever you choose shouldn't imply that students will climb these like a ladder each time. The attainment mark should be specific. Clearly a "Fail" grade would still flag up a concern for some intervention!!

To make this work, assessments should get progressively harder over the Key Stage, as we would expect them to. It needs to be made very clear to students and parents that this is a measure of specific attainment not progress. Students could then of course refer to their feedback on the assessment to help them work on improving. I know I have said this already, but it is worth repeating, it is perfectly possible for a student to get 75% on a maths test one week for example, and 65% the next, but still to have progressed in their mathematical understanding between those two points.

Reporting Progress
Progress should then be reported on as a separate element. Again, the reporting system should be unified across subjects as far as possible. The progress description should show a judgement over time and therefore be based on evidence from a wider range of sources than just assessment points. This might include teacher observations, contributions in class, students' ability to answer questions, the quality of their book work etc. etc. etc. Students' understanding of their progress should be embedded in the formative feedback they have received across the year. Trusting teachers' professional judgment is key here. 

Progress should be reported in a descriptive way to match its function as a measure over time. For example, student progress might be described as "None", "Slow", "Good" or "Rapid". Now if all of this sounds a bit woolly, it is worth bearing in mind that this is what Ofsted inspectors do EVERY TIME they visit a lesson. They sit in for 20 minutes, use the evidence from students and books, and make a judgment on the progress of students. If an inspector can do this in 20 minutes, I think a teacher can manage after spending 20, 40 or even 60 hours with students. 

The key to making this work would be to clearly define what progress might look like for students so that teachers can decide on the most appropriate descriptions of students progress, and of course so parents might understand what it means. This is the measure which should really flag up those students whose progress is not quite up to scratch, or who seem to be struggling.

Seeing as Ofsted already make these judgments about lessons, I began by looking at the criteria inspectors use to judge progress. Here is what I found in the most recent Ofsted handbook:

The key objective of lesson observations is to evaluate the quality of teaching and its contribution to learning...Inspectors should ensure that they:
* gather evidence about how well individual pupils and particular groups of pupils are learning and making progress...

Inspectors’ direct observation must be supplemented by a range of other evidence to enable inspectors to evaluate...the impact that teaching has had on pupils’ learning over time. Such additional evidence may include:       
* discussions with pupils about the work they have undertaken...
* scrutiny of pupils’ work, with particular attention to...pupils’ effort and success in completing their work and the progress they make over a period of time.* the quality and rigour of assessment...
* any analysis of robust progress data presented by the school, including information provided by external organisations

If you will excuse me paraphrasing here, in short, Ofsted don't actually have a definition of what progress actually looks like in a classroom. The only thing they have which describes progress, is when they talk about students making 3+ levels of progress from KS2-4, but this is based on attainment data and not progress. For the most part it seems that the average Ofsted inspector comes into a lesson, talks to some kids, looks at some books and then decides if this shows progress or not!! A science it ain't!

Now this is excellent news because it means that schools which actually define what progress might look like for students will be doing more than even the schools inspectorate themselves. Again, I have put a proposal below. This is by no means a finished model and requires A LOT of refining. However, I think it outlines generally what I mean when I talk about pupils making progress in my lessons. The purpose of this is to be descriptive rather than explicit - for explicit feedback, students would need to look back at formative feedback on work and in lessons.

The GCSE Issue

The final issue which might be of some concern to schools is the issue of GCSE uptake. Will a progress grade and an attainment mark be enough for students to make an informed decision? 

There are two possible ways of tackling this. Firstly, and probably the best solution in my opinion, teachers could speak frankly to parents at a set evening about their child's likley chances in a subject. Alternatively, teachers could be asked to make a banded prediction for a student towards the end of Year 9. For example you might note that you would expect a student is capable of getting A*-B, C+, B-C or struggle to hit a C. This may seem somewhat arbitrary, but it is no less so than students basing their GCSE choices on the KS3 levels farce. 

In Conclusion (finally)

I wanted to finish by revisiting my points from the beginning of this piece (now doesn't that feel a long time ago?!?). 

• We urgently need to create a better system of assessment and reporting for use in schools. The KS3 Levels system has been perverted from its original purpose and has no credibility as a model for measuring progress.
  
• We must prevent the conflation of measures of attainment with descriptions of progress - the two are not the same thing.
• We have to present the best cases for creating a better system and get those to our SMT as soon as possible.

Let me draw a (potentially rather strained) analogy. The removal of Levels was the storming of the Bastille. The prison in which we were held was thrown open and we were free to leave. But the initial euphoria of the new Republic has been tarnished as some old enemies rear their heads: Ofsted, progress, reporting, the choices other subjects have made about their progression models. We therefore find ourselves suffering something of a Bourbon Restoration. The Levels might be gone, but it seems that many schools are intent on either keeping them at a whole school level or modifying them only slightly. We now need our 1848 moment - we need to remove the last vestiges of the Ancien Regime and ensure there is no chance it can return (I am choosing to ignore Napoleon III in this analogy). Now is the time to pull down the old systems of progress and assessment and replace them with somethign worthwhile. Vive la revolution!!

Any comments or thoughts on this piece would be most welcome.

Mr F