Those in the sector hoping for an easing off when it comes to the bad news stories will doubtless be disappointed to see today’s coverage of centre-right think tank Reform’s report on grade inflation. For fans of the genre it’s a special bit of work – it’s the same old stats, an enormously cynical narrative, and a set of recommendations so daft as to be laughable.
A history lesson
First, we’re transported back to the heady days of britpop, free education and no restriction on taking another degree – “In the mid-1990s there was no detectable ‘grade inflation’ at all”. We’re then yanked into the low fees noughties, where “from 1997 to 2009 the proportion of Firsts almost doubled from 7 to 13 per cent”. And what happened once we stiffed them with £50k graduation debt? “In just seven years since 2010 the proportion of firsts has doubled again from 13 to 26 per cent”. Scandalous!
“Establishing causality is problematic, yet the correlational evidence suggests that when tuition fees rise, so does the proportion of top degree outcomes”. Maybe that big investment means they’re working harder. Maybe more students are working hard to achieve the standard. Maybe teaching has improved, and assessment has become more diverse. Maybe more students are taking resists. After all, “inflation itself must be driven by factors that directly translate into universities awarding higher marks”.
Trouble is, the report then goes on to look at all the other reasons that the sector has cooked up for the miracle. A pro-VC from UEA is mocked for citing improved entry qualifications, though without mentioning the student to staff ratio shift from 18:1 to 13:1 in the rest of his quote. Degree algorithm fiddling is cited, recycling a debunked quote. And without any reference to hard work or student support or assessment techniques, it then finds a handful of academics’ anecdotes to say they’ve been pressured to lower standards. Cue the A-levels chorus of “we worked harder and so did students” from the sector, falling on deaf ears in the press and the think tanks.
Premium mint wafers for all!
The row over grade inflation tell us something about massification. It is probably true that expansion means that more people are better educated than ever before. But it’s probably simultaneously true that that a “first” isn’t as robust a signaller or sorter of standards as it was, either within an HEI or across the sector. It’s like selling After Eights in Poundland – it’s an accepted good that more of society gets to experience that treat, but the elite will bemoan that its signalling power is now diminished, and the press will notice when the box is smaller.
Central to this set of sector growing pains is the use and abuse of metrics. “Grade inflation” is posited in the Reform report (and in TEF) as evidence of gaming – a sign that standards are slipping as providers scramble to climb the tables that have included the percentage of “good honours” as a generator of points for years. Meanwhile a reduction in “drop out” (or “non continuation”) is posited in TEF as a good thing – a sign of increased effort from students and improved performance from providers helping students achieve their objectives.
The trouble is that you can play both of these metrics the opposite way. An improvement in success rates can easily be justified (and in some cases evidenced) through enhanced support and assessment innovation, particularly when targeted at those on the wrong end of achievement gaps in a more diverse study body. And an improvement in drop-out rates can be framed as a sign of slipping standards as providers do all they can to cling on to the funding units they need to get through the demographic dip set to continue into the 2020s.
Almost any outcome metric is susceptible to a cynical suspicion of gaming – fiddling with the inputs or the thresholds instead of working to actually improve performance. If we’re being honest, in any institution it’s almost certainly simultaneously both an unfair accusation and a bit of a truism. And in some cases, what some would regard as gaming is actually the intention of the metric in the first place. Does DfE want providers running humanities courses with poor LEO numbers to improve links with employers, or inject skills content, or ditch the provision in favour of technical and vocational? Or all three?
Without condition
The danger is that we sleepwalk into crisis after crisis. Take tariff discounts. Doing so through results day sleights has long been a practice, but risky – so unconditional offers are now rife throughout the sector. With posh school sixth form tutors now reporting that a critical mass of their seventeen year olds are now holding unconditionals and exhibiting behavioural issues as a result, it’s only a matter of time before this becomes a scandal with a metric in the TEF too. How can selective universities defend selection in theory, with all the inequalities that it’s a proxy for, if they’re abandoning it in practice?
When it comes to wonks, it’s all too easy to argue that the debate about metrics gaming looks at issues through the wrong end of the telescope. Moans about TEF argue that it’s an arbitrary classification that uses a weird algorithm where the right background can get you bumped up if you’re on the border. It doesn’t measure the things that matter, doesn’t compare like with like, and it’s a pretty useless signalling mechanism. I’ve lost count of the critiques of TEF from indignant sector types who rubbish the excellence framework while clinging to the UK’s baffling degree classification system with no hint of irony. Now you know how students feel.
But the reality is that the thrust of government policy of any stripe – that more of us can have what was once the preserve of a few – is unlikely to change, especially when Bob Burgess’ review of the system over a decade ago recognised that the “honours degree… is the core product of the UK higher education system”. Delivering that “product” en masse will always involve economies of scale that cause the actions of competitive actors to be playable both ways. The question in the mess is what – or indeed who – can act as the counterbalance to the cynicism, and that has to mean more than just telling the optimistic side of the story. Someone has to spot that Audi is fiddling the emissions numbers before it happens. DfE often argues that it’s for providers and the sector to show care or restraint, yet the one bit of the Reform report that does resonate is that it’s not at all clear that any of OfS, QAA, Advance HE or UUK are set up, ready or willing to spot, call out and prevent gaming when it’s happening.
A case for quality enhancement
To avoid falling into one of the twin traps of resisting measuring things at all or being caught out by another scandal, we should talk more about innovation in teaching learning and assessment, seek as a sector to understand institutional behaviour and be more open when we’re doing good. One of the miserable things about TEF talk is how much focus there is on the system and the game – but hardly anyone talks about how important, for example, decent assessment and feedback is to learning to students. And this is about more than PR – the culture of competitive secrecy we’ve ended up with (resulting in one provider last year refusing my FOI request for their mental health strategy on the basis of commercial confidentiality) means that it’s genuinely difficult to find out what other providers are doing to shift the needle on their bit of the sector’s continued NSS achilles heel. Reframing this kind of information as “public” – enforcing the expectation buried in E2 of the regulatory framework that “decisions are taken in public” and allowing the collective sector to tell a different tale on metrics and what they tell us about the student experience – will become crucial in the years ahead.
If we were bold enough to recognise that you can cause positive shifts in institutional behaviour without having to fling revealing but faulty metrics data at loan voucher holding applicants, we might get somewhere. Imagine a TEF where the metrics deliberately change every year. Imagine an annual process where ministers, sector leaders and students agree on a set of benchmarked metrics to be judged four years out and there was collective, open, innovation and effort at sector improvement. Imagine if this process were to be repeated at institutional level, where regulators would require corporate and academic governance to set some metric targets locally in response to contexts and distinctive missions. Medals and awards – which will still be prized – might just start to look like rewards for effective leadership, additional student effort and innovation in delivery in the student interest, which is probably what we wanted all along.
What is clear is that the report’s central proposal to fix all of this, a “Designated Assessment Body” for “each subject” that will include “a single, national assessment lasting approximately 3-4 hours for each degree course” is not one that is likely to be adopted any time soon. But without rapidity, humility, subtlety and smarts, the eventual outcome might be much much worse for UK HE.
Quick summary of previous responses, querying the assumption that grade inflation is necessarily bad.
1) If attainment gaps have closed (eg male/female gap, affluent/deprived student background gap, white/ethnic minority gap) by the under-achieving group catching up with the higher-achieving group, grade inflation is probably a positive thing.
2) If average marks awarded have risen (ie it is not just the case that the degree classification proportions have shifted), and if positive skew in the distribution has not been replaced with negative skew, this indicates that grade inflation is not the only potential explanation.
3) Even if grade inflation as conventionally understood has occurred, the cure could be worse than the disease. The cure could take the form of students undermining each other rather than working collaboratively, seeking to manipulate or complain against lecturers, students motivated by mark gain rather than a desire to learn (not the same thing), even higher levels of mental health anxiety than present.
4) In most subjects, students achieving first class degrees do not have better career outcomes than students with lower second class degrees. This suggests that employers do not rely on degree class as a signal and have developed effective recruiting mechanisms.
“employers […] have developed effective recruiting mechanisms.”
Oh no I was with you right until this point! As much as I’m inclined to say that degree awards” purpose is not to differentiate graduates in the labour market, I’m deeply skeptical that the labour market has hit on a better model. Surely the replication of existing inequalities in access to the professions indicates exactly the opposite?
Bradbury Smith’s summary is succinct – and its likely that attainment gaps are closing (at least in some HE institutions), in which case ‘grade inflation’ is a positive thing. The real issue is not that ‘fiddling’ with degree algorithms is causing grade inflation (or more correctly – classification inflation), its that degree algorithms that discount (i.e. ignore the lowest marks) and use differential weighting (i.e. place a greater weight on the final year marks) advantage one group of students – that is to say: they are unfair. Such algorithms reward inconsistent mark achievement, while those students whose marks are consistent are, by default, not advantaged. In its current framing the debate on ‘grade inflation’ completely (perhaps conveniently) ignores the very real issue of equity.
I agree with David Allen. I’d also like to add the point that degree algorithm fiddling devalues the degrees of previous cohorts of graduates. If someone is applying for a job/course and has a 2i, while someone a year younger applying for the same job has a 1st in the same subject from the same university despite having lower marks, the first student will quite justifiably feel her/his university has defrauded her/him. From my experience of looking at degree transcripts of graduates applying for medicine as a second degree it is no exaggeration to say that a student with a 1st might only have achieved a 2ii with the same module marks at a different institution. Allowing students who have effectively failed a level-5 or 6 module to graduate with first-class honours – as happens at some institutions – brings honours degrees into disrepute.
In many subjects final year modules are harder conceptually and may well be marked harder. To give them more weight advantages the better students, which is perhaps the desired outcome.
Who even knew you could get After Eights at Poundland?