Dream on dreamer – TEF gets in the way

One of the things that has been worrying me for some time is the dependency we have in the sector for regulating and judging provision by taking the “provider” as the object of focus. Let me explain.

As a child, I wanted to be a keeper at Dudley Zoo, and a recurring dream of mine is that I go back to the Midlands and do part time HND in Animal Management. I looked at it for real the other day. Halesowen College popped up on Unistats, and I was pleased to see that it has TEF Silver. “Very high proportions of students continue with their studies and progress to employment or highly skilled employment or further study”, said the provider statement, which was very encouraging.

But me being me, I also decided to look at the OfS register to check that Halesowen College was all ticketyboo. Imagine my despondency. “Condition of registration imposed”, it said. “B3… Deliver a material improvement in continuation rates for other undergraduate students”. Never had a dream come true, anyway.

Thinking it through

There are lots of reasons for this apparent discrepancy that I won’t cover here. But it did get me thinking. I could find plenty of large universities with single courses that have more students enrolled in them than Halesowen altogether. Those courses can have terrible continuation rates, but won’t get picked up by OfS for a registration condition because they can hide inside vast averages. Regulation that acts in the student interest ought to weight risks to individual students equally – but it doesn’t. The bigger the institution, the worse a pocket of provision can be – without anyone really noticing.

That is one of the signature reasons for the introduction and piloting of subject-level TEF, and on that, I have a resultant prediction. Imagine you’re a DFE official, or a minister, or a pro vice chancellor – or Dame Shirley herself. There are multiple critiques of the TEF – the metrics weightings, the politics, the cost, the use of NSS – but let’s say that on balance you think it should stay, with a tweak or too.

Staring at the medals

Now imagine that at the end of a subject pilot, you’re staring at a large, “TEF Gold” institution that at subject level contains thousands of students on “Bronze” provision. Then you look to your right, and on another table in your air-conditioned meeting room, you see a tiny Bronze institution – trying hard in tough circumstances, that in totality has fewer students than one of the Bronze subject areas in your “TEF Gold” megaversity.

You know that students in a position to choose will browse unistats, look at Localton college and dismiss it out of hand – while yearning for the Gold medallioned magic of Megaville university. And at that moment, you’ll also know something else – that it simply ceases to be morally justifiable to retain institutional TEF now that those subject level medallions have been piloted. Because believe it or not, from an applicant perspective institutional TEF is even more meaningless than subject TEF.

Choose a door

You would then have options. You could argue (as is the current plan) to keep both – but how on earth would students interpret a Bronze course at a Gold institution when the latter uses almost the same metrics, only less specific to your course? You could argue that both should exist, but with completely separate metrics – but given there’s no magic blueprint for what is devolved to academic departments and what’s run centrally, that won’t work either.

You could pursue subject level on its own, but the more you look at benchmarking, and statistical significance, and the basket of measures’ relevance to all courses (let alone its relevance to all students), the more you think the hassle outweighs the effort – not least because newspapers do a better job at remixing the metrics than you do. And then it dawns on you that some academic departments in some universities will straddle your subject groupings, and you’ll realise that there isn’t the room in their school office, their messaging or their accountability systems for all three medals to apply to that school all at once.

So where does that leave you? You probably conclude that getting data out there in ways that applicants can use – especially through a new Unistats that the regulator is already planning – is more helpful than telling an applicant that their chosen course is in an institution with 40k students whose averages are “Gold”. You do your level best to get as much data as possible into the hands of students’ union officers and reps, who will spot pockets of provision that need real attention and will say so back – publically.

You amend the regulatory framework to require institutions to publically reflect on NSS scores and LEO data, and publish an action plan that gets monitored – in roughly the same way that OfS does now on access and participation. You search for innovative and interesting ways to incentivise and judge excellent teaching – because that’s what students who think about “teaching excellence” actually want you to do. And you commit to a focus on teaching enhancement, student partnership and institutional collaboration. Obviously.

And then you sleep soundly – safe in the knowledge that you’ve recommended a “framework” that is cheaper, more useful, more meaningful and more impactful than the one we have now. After all, you’ve got to have a dream – if you don’t have a dream, how are you gonna have a dream come true?