I was accused of displaying “withering cynicism” recently, something which apparently is now considered “tiresome”. The accusation was a fair one, not least that the topic under discussion was whether the Teaching Excellence and Student Outcomes Framework (TEF) was something offering material benefit to the higher education sector.
While I’m not going to relent on my assertion that TEF doesn’t measure teaching, and that its outcomes are based too much on creative writing than on evidence, I want first to offer something more positive.
Adding value
The launch of the latest TEF consultation, on the exercise’s subject-level variant, has resurfaced the spectre of salary outcomes from the Longitudinal Education Outcomes (LEO) dataset being used as a direct measure of “quality”. The problem here – as has been well rehearsed – is that absolute salary is not a measure of the quality of a graduate’s education. But, we could get closer to something meaningful.
I have argued on Wonkhe before that the real value in TEF is its benchmarked performance measures, something which doesn’t show up in traditional league tables. LEO data – if benchmarked for students’ social background, gender, ethnicity, region of employment and other material impacts on earnings – could be used one proxy for the added value of a degree course from any given institution. The rich kids will always be fine: why don’t we have measures which incentivise institutions to do the most to build social capital for their most disadvantaged students?
There’s a wealth of information available to prospective undergraduates about universities. The National Student Survey (NSS) has for ten years been a valuable source of information. And it has led to major improvements across providers. NSS is, currently, just for undergrads though. With the establishment of OfS, what will happen to the NSS for taught postgraduates under development by HEFCE?. OfS is explicit about its desire to work for “all students… undergraduate and postgraduate” but measures for supporting PG students are hard to find in the regulatory framework. Worse still is the paucity of concern for research students about whom many in the sector expressed concern that they will be forgotten in the split of HEFCE to form OfS and UK Research and Innovation. Surely now’s the time for some more attention on postgraduates?
A little honesty
OfS has adopted the troublesome child that is TEF (in which participation will be compulsory for providers with more than 500 students), and defined the exercise as “a scheme for recognising excellent teaching.” The casual reuse of “TEF equals teaching” is unhelpful. First, it allows for lazy parallels to be drawn with metrics-driven teaching improvement in schools. Universities and schools are not the same things. They have different – valuable – missions, but to place such focus on teaching in higher education is to neglect the ecosystem which includes research, knowledge exchange, public engagement and so on.
I’m all for improving the quality of the student experience, and also for improving teaching. It’s important that there are structures to support this, like NSS. The rub, however, is that it is legitimate for the English HE sector to be anxious when its new regulator promises to be data led and then celebrates a deeply flawed exercise.
To labour my point: the diminishing role of NSS means that TEF is even less data led than it was. TEF is also a great example of duplication of data collection and reporting; the newly introduced grade inflation measures should surely be collected and reported by HESA than through the exercise. As an aside, OfS requires universities to publish a host of data – a product of the “transparency duty” – but wouldn’t it be better to have this all in one comparable place rather than presented in the many variants that individual providers will choose? And it’s still ridiculous to boil down the wealth of information about an institution into a three-tier medal system.
The case for cynicism
Wonkhe has been a platform for policy criticism; the most casual of readers will have noticed this basic fact. I would argue that informed critique – even with a cynic’s inclination – could be useful to those charged with making policy. It’s early days for OfS, and too soon to judge whether its promises of low burden and useful regulation will come to fruition. I hope that a little constructive engagement – even with cynics – will help it achieve that ambition.
I have been explaining to new staff at induction sessions that the Teaching Excellence Framework measures neither teaching nor excellence. It’s hard to see how it could be what it purports to be when you actually look at what’s measured. Even the best of them, the NSS, only gives you an indicator of how students feel about their experience at a single institution (which always makes comparisons between institutions something of a strange exercise). So to suggest cynicism seems somehow misplaced – how could you not be cynical about something so evidently daft?