The value of a national TEF is in enhancing university learning and teaching

It is great that the TEF review is finally out in the open and the government accepts most of my recommendations.

The government response from the Department for Education agrees that there should be a clear purpose for a revised TEF and that enhancement of provision should be the goal. This clarity of purpose makes redesign more straightforward.

The government also accepts the principles for redevelopment that the review proposed. A redesigned TEF should be transparent, relevant and robust. These are all important in multiple different ways.

Students should have a voice, but not just through the NSS

To give just one example of the transparency principle, I was concerned that the NSS was being used as a proxy for too many aspects of quality. Not only did this raise statistical concerns about multiple comparisons but it also used students’ impressions to infer absolute quality. Students’ judgements about their experience are important but they are not an absolute measure of quality. There is no absolute measure of educational quality.

We proposed that a revised TEF should be transparent about where students’ judgments fit into the assessment process and proposed an explicit “student satisfaction” aspect to the structure. The NSS has played an important part in the assessment of student experience and whatever the future of the current NSS, students’ views of their education should always be part of a national assessment of excellence.

The government has agreed the student voice is important but does not agree with the name “student satisfaction”. The response suggests an element called “student academic experience” but does not yet say what nationally comparable metric might be used. Our other proposal, that the submission process should enable the student body to contribute independently, would ensure a student voice, but this would not enable national comparisons. The student submission should not be considered an alternative to a nationally comparable metric of student report, such as the NSS, in the structure of the TEF framework.

Subject data drives useful conversations, but public ratings are not robust

The review made a significant number of other process and statistical recommendations, drawing on the tremendous work of the Office for National Statistics (ONS). The statistical risks in the TEF are at their most acute when making subject level ratings which is why we concluded that TEF should not proceed with public subject ratings. At the time of the review this was one of the most polarising considerations and I am pleased that the government has accepted this recommendation.

Although I, and the TEF advisory group, found undue risk in developing subject ratings, we were impressed by the consistently expressed view, (by lovers and haters of TEF) that the provision of subject level data with splits for the various benchmark characteristics had a value within institutions in identifying areas for improvement.

In particular, institutional leads for teaching and learning were clear that the benchmarked data enabled them to pull levers for change in their institutions. The data helped them gain institutional attention for areas that needed investment or improvement. We concluded that there was a value to enhancement, within institutions, of access to the granularity of information in the subject level data.

We also consider that the way in which providers identify and respond to their own variability in subject performance is a key part of their enhancement process and an assessment of this should be incorporated into the overall provider TEF ratings.

This should be the case for all universities. Even high performing universities may have pockets of less successful teaching and learning, and this must not be overlooked if all students are to thrive. I welcome the government response making clear that the TEF exercise, however it develops, should apply to all universities.

Institutional ownership of some metrics could help balance diversity and accountability

A message from all parts of UK higher education during the review was that “we are different”. Different HE providers have different missions and provide different learning experiences and opportunities for their different groups of students. There was also a strong message that formal teaching is only one part of the overall range of learning opportunities that comprise high quality HE.

This difference of mission across different providers, with multiple different kinds of learning outcomes makes it impossible to judge all provision against a single set of metrics. Students want a choice, and the country (and the world) needs a range of provision.

This is why we recommended that a redesigned TEF should request institutionally determined measures, to enable each provider to demonstrate how they measure their own performance against their specific mission. This would supplement nationally comparable metrics.

Overall ratings should be derived from both qualitative and quantitative data. This enables institutions to articulate and demonstrate how they create an excellent educational environment and how they facilitate their students’ learning. The government agrees that qualitative and quantitative data are important. An overall judgement to determine the final ratings also enables contextual factors, such as the devastating Covid-19 pandemic, which may have differential impacts across the sector, to be taken into account.

Taking the TEF forward

We have the Higher Education and Research Act (HERA) 2017 to thank for the unusual opportunity to involve the sector in a review of policy at a relatively early stage of its development.

The expertise and care that the TEF advisory group brought to the multiple strands of the review; the listening sessions, the call for views, the statistical analysis from the ONS, the survey of international experts, the British Council report, the surveys from UCAS of current and prospective applicants and our interviews with employers, provide a rich source of data and ideas for all interested in educational enhancement. I can only touch on some of the issues we raised and addressed. I hope you will enjoy reading the report itself!

The government has asked the Office for Students to take forward the redevelopment of TEF in the light of all this evidence. I welcome that and hope the sector, which had such a significant input into the evidence base, can continue to share its expertise with OfS as it consults on practical proposals to implement the changes.