Guest contributor Thomas Bearman on the UK government’s recent grading algorithm, what went wrong, and what can be learned from it.
Some Roar articles have covered the recent scandal with the grading algorithm that caused A-Level and GCSE results to plummet around the country. In England, Wales, and Northern Ireland, A level grades dropped by 40% and 2 million GCSE results were downgraded. The question remains, however, of how and why this happened. What went wrong with the algorithm and how did no-one realise sooner?
The grading algorithm is a complex on the surface but simplistic enough once you dig into it. At the start of the pandemic, schools were told to provide estimated grades for students and to rank each in a class from best to worst.
Ofqual then created a database of past predicted grades from the last three years and their achieved grades from the last three years. They calculated the difference between the previous distributions (I.e. the between the previous predicted years’ grades, and the previous actual grades), and assigned grades based on the ranking.
In practice, say that you were ranked 10th in your class by your teacher, and in the previous 3 years the 10th highest student had achieved a B. You would then be given a B.
Why not instead use the given grades from the teacher? Surely people who know the students would have the best grasp on the situation?
Well according to Ofqual, it would have produced unacceptable grade inflation, seeing A* grade in A levels rise from 7.9% of all grades to 13.9%. This would have made the results useless compared to other cohorts and likely cause many universities problems in the selection process.
Some might ask instead, why not sit exams in quarantine somehow? Surely that would at least give some practical idea as to the student’s level of achievement? Well, Ofqual claims that: “Our initial advice to the Secretary of State was that the best way to handle this was to try to hold exams in a socially distanced manner.â€
However, Ofqual also claimed that the Department for Education (DfE) took the decision to cancel exams without properly consulting them. Perhaps, given some schools had stopped teaching as soon as the quarantine started, such measures would be impossible. That was the position taken by John Bald, the Vice-President of the Conservative Education Society.
This grading algorithm was left as the only plausible solution. So, the question now remains, were there any signs of it going quite this badly? Ofqual’s report on the issue seems to say that they were to some extent ignorant: “… evidence from our analysis of calculated grades shows no evidence of systemic bias in the grades awarded this yearâ€
However, it might be more pertinent, in this case, to pin on the addendum. There was no example of any additional systemic bias in the grades. The inherent bias in the grading algorithm was left as it was. While Ofqual did not add in any additional bias, this existing bias in the system was still there.
This is not to say there was no general awareness of potential problems bias in these grades might cause. Ofqual allegedly briefed Number Ten a week before the incident, and again on the 7th of August, notifying them about potential dangers. The Royal Society was not the only one worried about results. Several writers remarked it would likely be difficult for schools to accept this algorithm.
Although, this was not before Ofqual themselves were warned by the Royal Statistical society about the potential problems with their work and lack of transparency, after they were forced to decline work on the algorithm due to the nature of the non-disclosure agreement handed to them by Ofqual.
It seems this issue, despite being known, was allowed to snowball until it hit the headlines and created a public outcry. So, what has the response been to this wave of anger?
Starting with concrete examples, Ofqual chief Sally Collier, and Johnathan Slater, one of the DfE’s most senior civil servants, have both stepped down. The Government has also launched the Exam Support Service to assist those retaking exams in autumn. The Welsh Education Minister, Kirsty Williams, has also confirmed that an independent review of the exams will take place, with full results expected in December.
Gavin Williamson and Ofqual also announced that the algorithm grades were null and void, and have since reverted to using teacher assessed grades. This has not been viewed favourably by universities, who have complained of grade inflation that has led to a far bigger intake than planned. This lead to universities requesting financial support for the large intake and for social-distancing measures.
In response, the UK Government has partially dropped the cap on UK students and the Scottish Government said it will meet in full the costs of the extra students.
As for the less practical responses, as can be expected from this government, blame has been laid everywhere but the door of Mr Williamson. From Ofqual to the DfE, blame has been thrown around wildly. Many of the papers seem to be focusing the fire on Williamson, but despite many calls for his resignation, he seems to have escaped relatively unscathed for now.
So, what can we learn from this?
Firstly, that Government pushes for more technological solutions seem to be increasingly doomed to fail. Between the algorithm, the track and trace app, and a website designed to help people find their nearest testing centre sending them to the other side of the country, it seems unlikely that this government will continue on this path.
The government also faces increasing fractiousness within their own halls. With the repeated blaming of civil servants and departments, tension must be brewing. This is coupled with frequent and blatant refusals by ministers to take the blame for their actions, seen with Dominic Cummings, Boris Johnson, Mr Williamson’s recent catastrophes not having seen reprisal, and the government’s repeated shunning of experts and the Civil Service itself.
Finally, the continuity of poor planning is beginning to show. The government’s latest U-turns have affected the futures of thousands of students. Their decisions, and lack thereof, will not only shape the next election, but also governmental relations with education for years to come.
