The 2023 Teaching Excellence Framework (TEF) results are now published for the majority of the 228 providers that took part.
This makes it possible for me as the TEF chair to begin to reflect on the outcomes and the experience of the revamped TEF, and the longer term implications for the sector.
The new TEF
In the first TEF, providers were awarded a single rating – Gold, Silver or Bronze. Following Shirley Pearce’s independent review of the TEF, which drew on inputs from across the sector and an expert advisory group, many aspects of the TEF were overhauled.
This makes the 2023 exercise significantly different both in outcomes and process. That single rating has been replaced by a profile with separate ratings for student experience, student outcomes and the provider’s overall effectiveness. A ‘requires improvement’ judgement has been added to the Gold, Silver and Bronze judgements.
A student submission has been considered alongside institutional submissions and the treatment of statistical indicators has been revamped. The TEF has explicitly engaged with approaches to the assessment and evaluation of students’ educational gain as a result of their studies. The result is a 2023 TEF which is more sophisticated, more fine grained and yet more responsive to the ways in which very different providers deliver institutional missions.
Distributed excellence
Other blogs and commentaries this week will explore the pattern of outcomes, but the overwhelming picture which emerges from the 2023 TEF is of a very diverse sector in which excellence is broadly distributed between different types of provider and different parts of England. The 2023 TEF has identified the relationships between institutional mission, institutional policies, institutional practices and student cohorts. It has done that in ways which are sensitive to the very different practices of, say, a research-intensive high tariff multi-faculty university, a specialist conservatoire placing tight focus on the development of high level performance skills, a further education college working in close partnership with employers – and so on.
Inevitably, commentators will want to dig into the patterns of ratings across a diverse sector, but they should also step back and consider the picture as a whole – of diverse providers working in so many very different ways.
The 2023 TEF has been an enormous undertaking. As TEF chair, I have been extremely grateful to those who have made the assessment possible. I have been fortunate to work with two outstanding deputy chairs – Professor April McMahon, Vice President at the University of Manchester and Obi Okereke, formerly student sabbatical officer at the University of Coventry and now the university’s Project Manager – Student Experience. I have also been simply awed by the thoroughness and assiduousness of the 64 TEF panellists who have worked with incredible care through indicators, institutional submissions and student submissions for 228 providers.
TEF works
Every provider’s submission was treated with care and thoroughness, applying the detailed assessment guidance – so often in panel discussions attention would be drawn to the implications of a specific clause in relation to an individual provider. I am not naïve enough to suppose that every provider will have enthusiastically received its ratings, but all providers should know that the ratings were the result of deep analysis, extensive discussion and repeated review. I do know, from conversations across the sector, that whilst some providers are not happy with their grades, all are struck by the professionalism and thoroughness of the institution-specific feedback. And I have been so grateful to the OfS TEF team, who have worked so hard to deliver a large scale and complex assessment process to time and to specification: all of them concerned to ensure that the TEF delivers what it should deliver – a careful analysis of teaching excellence and student outcomes across the sector.
The TEF results should and can raise the bar once again for thinking about teaching, student experience and student outcomes. They should do this in some quite specific ways. They should focus thinking on the relationship between strategic improvement initiatives, of which there are striking examples across the sector, and the outcomes for all students – time and again, the TEF panel focused on the assessment criterion which asked them to look at the outcomes and experiences for all students. They should focus thinking on institutional strategies for educational gain and the ways in which those strategies are evaluated and assessed. They should focus attention on the entire student life cycle, on the relationships between student experience and student outcomes, and on places where those diverge, and diverge for specific groups of students. Perhaps most of all, the TEF results should encourage different providers to extend the range of their learning: there is excellence in every part of the sector, and we all have a moral responsibility to ensure that we open our minds to learn from institutions which are very different.
That extensive changes were made following Prof Pearce’s independent review of TEF were no surprise to anyone who possesses any statistical training, not least the experts at the Royal Statistical Society who thoroughly analysed, and objected to, much of the original TEF methodology. To their eternal credit, as well as contributing extensively to Prof Pearce’s review (https://rss.org.uk/RSS/media/File-library/Policy/Higher%20education%20policy/2019/RSS_Evidence_to_the_TEF_consultation_Feb2019.pdf?ext=.pdf), they said as much publicly: https://rss.org.uk/RSS/media/File-library/Policy/Higher%20education%20policy/2019/TEF-letter-Ed-Humpherson-March-2019.pdf?ext=.pdf.
As the RSS pointed out in their feedback to OfS on the current TEF proposals (https://rss.org.uk/RSS/media/File-library/Policy/2022/RSS_RESPONSE_OFFICE_FOR_STUDENTS_CONSULTATION_ON_TEACHING_EXCELLENCE_FRAMEWORK_MARCH_2022.pdf): “the very existence of TEF may be counterproductive to its aims.” In more granular detail, they point out strongly that there is insufficient attention paid to, or guidance offered upon, the use of multiple comparisons of metrics in TEF submissions and TEF panel judgements. The implication is that TEF metrics are statistically unsound. Again.
More obviously (and less substantively), RSS highlighted that Prof Pearce’s independent review of TEF strongly argued against the use of a gold/silver/bronze award scheme. While a simplistic ‘tabloid-friendly’ scoring system has obvious benefits to politicians and, unfortunately, university marketing departments, it is misleading at best. The fact that only two universities received a bronze in the current awards means that one could argue with some justification on those grounds alone that they are not particularly meaningful. More important though is the more compelling argument that there is a collective public silence from universities about a lack of academic rigour and meaning in a prominent area of public policy concerned with education. Quite the opposite of what universities should contribute to society.