The strategic imperative of the TEF was to achieve long-term, systemic, cultural change of the higher education system, providing a permanent counter-balance to the distorting impact of the REF and its predecessors.
This means creating a TEF that is embedded in a sustainable, consistent form, that does not change on an annual basis, focused upon the fundamental priorities of high quality teaching and strong student outcomes. My submission to the Independent Review of the TEF calls for the review to focus on delivering the essential elements of the TEF needed to achieve this goal.
Goals and essential elements
The two principal goals – driving up the standard of teaching and informing students – are fundamentally linked. The REF is responsible for allocating billions of pounds of research funding. If the TEF is to be an effective counter-balance to the REF, it must have a similar impact on matters of direct, material impact to providers. In the current higher education system of minimal grant and no student number caps, this can only be done by impacting reputation and thereby student choice. To ask which goal is more important is a false dichotomy: removing the reputational and informing students element of the TEF would simultaneously destroy its ability to drive up the standard of teaching.
Though the details of TEF may change, there are a number of essential elements that are fundamental to achieving its goals. These are:
- Clear and meaningful differentiation with at least three different ratings. Only clear differentiation can deliver reputational impact and thus the incentive to improve teaching.
- Outcomes, not process, focused. This ensures that the results reflect and measure real-world outcomes.
- Metrics informed but not metrics determined. The strength of the TEF is in the interaction between the metrics and the panel’s judgement to deliver a balanced outcome.
- Benchmarking. Only by benchmarking can high-tariff and low-tariff providers be judged on a level playing field.
The TEF has become overcomplicated. The Independent Review should aim to strip out unnecessary complexity and restore simplicity to the TEF, reducing the number of metrics to 5-6, eliminating supplementary guidance and removing little-used parts of the assessor guidance and contextual evidence. The strength of the TEF is in the panel and they should be trusted and empowered to deal with nuance, borderline cases and exceptions.
Although it is essential to retain clear, differentiated ratings, the names of these ratings are less important. If a decision was taken to move away from Gold, Silver and Bronze, it would be most conducive to public understanding if the rating names were aligned with those used by Ofsted.
The use of robust outcome-focused metrics is a fundamental element of TEF. The wider public rightly see completion rates and progression to highly skilled employment as important measures of university quality and it is entirely appropriate that they should form a major element of any performance framework.
The precise combination and weighting of metrics used is of secondary importance to maintaining the essential elements set out above, provided they are robust, outcomes-focused and suitably benchmarked. However, some recommendations based on the current methodology are:
- There should be no more than five or six core metrics and no supplementary metrics.
- The NSS should ideally account for between a quarter and a third of the total metric weighting.
- All forms of further study, including progression to professional study, should be classed as positive outcomes.
- The time-period assessed by LEO should be moved to align more closely with that assessed by Graduate Outcomes.
- The grade inflation metric should be removed and other non-essential metrics, such the proposed metric on attainment gaps, should not be added.
- Absolute continuation rates lower than 90% and rates of progression to highly skilled employment or further study lower than 60% should attract an automatic negative flag.
- The differing ways in which providers relate to their regions mean there is no easy way to benchmark by region without introducing new distortions.
Subject TEF must not be “big, bossy and bureaucratic”
The principle behind implementing subject TEF remains valid. Performance by subject varies significantly within individual providers. Repeated research has shown that choice of subject is of high importance to applicants when choosing where to study and over 80% of applicants said that subject TEF would be useful to them in choosing where to study.
Nevertheless, the cost/benefit analysis of moving to subject TEF was predicated upon a methodology being found that did not require the assessment of all subjects. The pilots have found that neither model A nor model B were suitable robust. However, the cost of implementing the currently proposed model, in which all subjects are assessed, is a matter of concern and does not meet Jo Johnson’s commitment that the TEF would be “proportionate and light touch, not big, bossy and bureaucratic”.
I recommend that a minimum threshold be applied to subject TEF, such that only subjects with more students than that threshold are assessed. The aim should be to ensure that (a) at least 80% of students in the sector should be in assessed subjects and (b) the number of submissions should be reduced to between 1000–1500. This would also address the concern that the reliability of the metrics and therefore judgements diminishes for small subjects.
Such a change could reduce the cost of assessment by 65%-80%, a saving of up to £28m, while still providing subject information to over 80% of students. The remaining students would continue to be able to rely on provider ratings. Such a change would make the cost significantly more affordable for both the taxpayer and the sector, whilst preserving the majority of the benefits of subject TEF.