English school exams are fatally flawed

Covid-19 has brought exams firmly into attention with each of the UK regions struggling to find fair ways of assessing students when many of them have been forced to either miss lessons or receive a lesser education online. 

In all this the assumption has been that we must hold final exams if we can as they are the best way of assessing students, an assumption that led Gove to drop both continual assessment and AS levels.

It is an assumption that is wrong.

Exams are inaccurate

This is clearly true. I do not think that anyone expects that if a class of children took the equivalent exams, say a Geography GCSE, five days in a row that they would all get the same marks every time.

The question is not, therefore, whether exams are accurate or not but how inaccurate they are.

How inaccurate are they?

Surprisingly nobody seems to know. The best that I can find on my Google search is an Office of Qualifications and Examinations Regulation paper from 2010 that states, "The results showed that, for the GCE and GCSE units analysed, at least 89 per cent of all candidates with a particular grade (other than the highest or lowest grade) have true scores either in that grade or immediately adjacent".

That does not strike me as being particularly accurate and that statistical analysis is supported by the number of successful appeals against grades. Statistics published by Ofqual show there were 1,240 appeals in 2019, up from 857 in 2018. A total of 675 of the appeals were upheld, up from 406 the previous year – an increase of 66 per cent. These may be relatively small numbers but that is not the point, they are just further evidence that exams do not always give the correct result. 

That exams are inaccurate is not an opinion, it's a simple fact, yet most commentators talk as if they were perfect.

Sadly even education professionals make this mistake. When I was a school governor an LEA Inspector was concerned that our Teacher Assessment grades varied significantly from the results of the subsequent exams and the firm assumption was that the exams were right and the teachers were wrong. I asked them how accurate the exams were and they had no idea, to be honest I am not even sure that they understood the question.

Exams are just one data point

Many people will remember dealing with inaccurate measurements in school science lessons and part of the solution is simple, repeat the experiment many times and assess that actual results from the spread of results achieved.

So, if the class did take the equivalent exams five days in a row we would have five results for each child to work with and could, for example, take a simple average (mean or median, there are arguments for using both methods) of the five results and use that. This would not be perfect, five is still a small number, but it would be many times better than what we actually do.

Continual Assessment may have issues with personal biases etc., and I would not want to deny that, but it does provide a lot more data to make the final assessment from than a single exam ever can and there are ways to address the other concerns which would make some form of Continual Assessment a fairer method.

This, of course, is how assessment works in most workplaces during the Annual Appraisal process with employees having to cite examples of their work throughout the year as well as seeking feedback from multiple sources.

Conclusion

One data point that is measured imprecisely can only ever be right by chance (like a stopped clock) and that is why exams as used in English schools are fatally flawed.

I made it to Hatton Cross

Most of my long walks are ad hoc and unplanned but I have also done some well established routes like the Thames Path, Capital Ring and London Loop, all of which pass very close to where I live. Of these London Loop has proved most problematic with me losing my way in Hounslow Heath. Twice.

This day I was determined to complete London Loop Section 9 Kingston Bridge to Hatton Cross and I enlisted the help of Google Maps which has a copy of the route that I could copy to my maps and view on my phone.

The first part of the route was of my own making, partially because it was easier to cross the Thames at Teddington than at Kingston and partially because there were other places I wanted to go.

I did join London Loop for a while in Bushy Park but then I took bit of a detour which included a longer section of the River Crane Walk because it is pretty and I knew that I could get a coffee in Kneller Gardens.

I rejoined London Loop as it passes under the A316.

From there it was largely a matter of following River Crane as closely as possible.

I was more confident about Hounslow Heath this time and Google Maps did the job, it helped me to turn left, just above the "H" in "Heath" when I saw nothing on the ground to suggest this. That is where I went wrong the second time.

The other wiggling in Hounslow Heath was all part of the route and helps to show why it is possible for people like me to lose my way there. The first time I missed the doubling back above the "un".

I also managed to avoid my previous error of confusing Duke of Northumberland's River for River Crane just north of Hounslow Heath and I managed to keep on the correct path all the way to Hatton Cross.

This section was new to me and that helped it to be my favourite section on the day. The other reasons for liking were that I saw just two other people and for most of the time I was walking on a raised wooden path through trees. It was hard to believe that I was almost at Heathrow.

I was tempted to continue along River Crane but a dual carriageway, tube line and Heathrow all got in the way at the end of the section and it was a lot easier to hop on a 285 bus which took me almost all the way home.

My extended route was just over 20km and took me just under 4 hours. The slightly slower than usual pace, 5km/hour, was due to some route checking and correction along the way and, to be honest, a few pauses to take out Pokemon Go gyms.

River Crane very much dominates this section of London Loop, and rightly so as it is relentlessly pretty and universally well maintained. My decision to include more of if at the start was a good one even if that meant tramping further through the mean streets of Fulwell (and other indistinct suburbs) to do so.

Having cracked London Loop Section 9  I now have my sights on Sections 10 and 11 which will get me all the way to Uxbridge.