When we originally booked Dame Shirley Pearce – the independent reviewer of the Teaching Excellence and Outcomes Framework – to speak at Wonkfest, we were worried that her work would be old news by the time that the event rolled around.
How wrong we were. Despite Dame Shirley’s panel completing its comprehensive review on time earlier this year, the report has been sat languishing on ministers’ desks for months now, waiting for a slot to be laid before Parliament.
This inevitably meant that she was unable to share conclusions and recommendations at our session on the TEF at Wonkfest – but we did get some clues.
It’s all about the process
The session started with Dame Shirley sharing details of the process her panel went through to get to conclusions, and there’s an impressive collection of sub-projects that we’ll get to see when it comes out. The Office for National Statistics has had a close look at the metrics. The British Council has carried out some work on international attitudes to the exercise. And UCAS has been helpful in gathering the views of applicants on the usefulness of the exercise, and the information they need to make good choices.
There was also, naturally, a sector consultation – and the dominant theme here was tension. The panel had heard “diametrically opposed” views on TEF – some arguing that it generated a focus on teaching inside institutions, and others viewing it as a distracting burden. Some celebrated the use of outcomes metrics, others argued that it doesn’t actually measure “teaching”. And others suggested that the complexity in the benchmarked-metrics and narrative-context-submission model makes it fair – with others arguing that that complexity is what makes it burdensome and baffling.
It also sounds like we’ll get a reflection on learning gain; a look at how the metrics can be improved (LEO’s lack of geographical context was mentioned); some thoughts on the balance between absolute and relative values; a response to widespread student concerns about their input into the process; and some views on the “balance” between provider and subject-TEF. We appear to be looking at refinement here rather than anything particularly dramatic.
Digging in
The panel discussion explored some of the tensions in detail. Bath SU’s President Eve Alcock ably reminded us that both “learning” at university and the outcomes being measured can result from much more than the act of teaching, urging a focus on the wider student experience and student support. Eve was also worried that the choice of metrics for inclusion in TEF had the effect of making conversations about other things more difficult.
Bath Spa’s VC Sue Rigby urged the sector to “get over the overhead”, arguing that OFSTED inspections are much more draconian and arguably arbitrary than this exercise. Sue was keen that the exercise retained the ability to challenge the established order of university excellence, acting as a reputational disrupter.
HEPI’s Nick Hillman reminded us of where the exercise came from in the first place – cabinet level consternation (reinforced by research) that contact hours are down, but private study is up. He also argued that the reason that former HE minister Jo Johnson’s original claim that TEF would be “light touch, not big, bossy or bureaucratic” hasn’t come true is the sector’s tendency to overcomplicate questions, and he predicted that the Conservative manifesto will pledge to “root out” poor courses – TEF might (or might not) end up being an answer to that question.
Independence day
During the introduction to the session, Dame Shirley was keen to stress her independence – of DfE, OfS and of the sector itself. But independence is arguably where things get interesting next. We are told that DfE’s response to the review is ready to go – and there’s no guarantee that it will accept the panel’s recommendations. Then it has to land inside the regulator – and we know from September’s board papers that the view is that it is “for the OfS board to make final decisions about how it wanted the TEF to support the OfS’s regulatory processes”, and the board “will not be bound by the findings of the review”.
What we do know is that Gavin Williamson has already asked OfS “to publish subject level TEF in 2021”, implement a “new TEF model” and “seamlessly integrate” it OfS’s approach to the regulation of quality more broadly. In other words – both DfE and OfS may have already decided that some of Dame Shirley’s “tensions” are actually more like binary options. And it’s hard not to predict that the level of focus that subject-TEF will generate may well result in some outcomes, in some subjects, at some providers will soon be falling below the B3 bear baselines. TEF-aluminium, anyone?
When aluminium was first produced it was a precious wonder, perhaps prized above bronze, light, strong and exotic. Now it is cheap, unrivalled in many uses and everywhere.
I think the dig in “TEF-aluminium” is telling. If everyone can have it, it must be no good, no matter how well it works at the job it’s been given.