It will not surprise the attentive reader to learn that there are (significant) parts of the higher education sector which would like to see an end to the Teaching Excellence Framework before it even begins. For them, the political instability caused by the EU referendum and the redrawing of departmental boundaries in Whitehall provide an ideal opportunity to make polite – but pointed – suggestions that TEF should be a mere trial. Its future deployment should be put on hold pending proper, thorough, drawn-out evaluation. In translation, that might become: “See that long grass over there? Would you mind awfully kicking this unwanted nuisance in that direction?”
If you want to identify the holes in the government’s plans for year two of the TEF, you’d do well to read the Royal Statistical Society’s submission to the technical consultation. Year two is the exercise due to take place in the winter of 2016-2017, aiming to influence student choice for 2018 entry. You may recall that the exercise will result in judgements that providers meet expectations, are deemed excellent, or receive the top prize of an outstanding rating. And there are commendations available as extra bonus prizes.
I’d like to suggest that now that the ink is dry on the consultation submissions, it’s the time to look both forward, to the planned iterations of the TEF, and backward to the genesis of the exercise. That’s because TEF does actually have the potential to do some good. A controversial position in some circles, but hear me out.
Students apply for individual programmes. Yes, they might switch once they arrive at the institution. And yes, their eventual graduate prospects may be affected as much, or more, by the provider on their degree certificate than by the name of the award. But it’s the teaching experience as received, or excellence as judged, within the discipline which matters to the student’s experience.
The assessment of teaching – and with an emphasis of student satisfaction measures and employment outcomes, TEF is much more about the broadly-defined educational experience than it is solely teaching – at institutional level masks for many providers the wide variety of their provision. For the multi-disciplinary providers, there will inevitably be variability across academic units, levels of study, modes, delivery locations and over time. Delivering a judgement on the totality of a university’s portfolio has precisely the same deficiency – for supporting a student’s choice of programme – as league table rankings of overall institutional standing.
It is only when TEF reaches its intended maturity, from year four (for students entering in 2020), that there is potential for the exercise to realise material benefits (whether those benefits are worth the cost is a related but separate matter). By this point, debates over the rights and wrongs of the metrics should, we hope, have been concluded and institutions will have had time to practise how to game the system. Oh, wait, I mean learn from good practice and make interventions to improve students’ experience.
It’s also at this point that TEF will have the capacity to pass judgement on individual disciplines within a university portfolio. Now, I realise that disciplines and programmes are not the same thing. However, the discipline is much closer in proximity to the programme than the institution, and – in many providers – the accountability for delivering teaching rests at a disciplinary level.
From TEF year four, there will be parity for the small institutions and for specialist monotechnics, so that their issue of small numbers does not unduly hamper their participation in the exercise. For the initial rounds, there is a recognition that data sources aren’t robust for the smallest providers; this would likely still be true in later exercises, but at least it would be fair(er) across the board.
When there are ratings by discipline, there there is actually the capacity for TEF to improve teaching and the wider student experience. That’s because it will be more likely, I believe, for providers to make interventions where they identify deficiencies in the data, particularly when those deficiencies are public. It will take some time for all disciplines across all universities to be rated in full as there’s unlikely to be uniform participation, given that the TEF ratings last for three years. But when the exercise is at steady-state then the information to support prospective students’ choices should be available.
The journey from TEF one to four will be bumpy. There will be niggles, there may be legal challenges, there will be some justified winners and losers, and some unjustified. The real prize of TEF, that which might benefit prospective students, is the assessment of a quality measure at disciplinary level, and its publication. That is the point where meaningful comparison can be made. And, in even better news for prospective students, year four will be the first time that judgements are available on taught postgraduate study, an area for which there is a significant deficit in the quality of information available to prospective students.
Finally, let’s consider the mandate for implementing the TEF. The Conservative Party’s 2015 manifesto stated that the government “…will ensure that universities deliver the best possible value for money to students: we will introduce a framework to recognise universities offering the highest teaching quality…” For the first three years of the TEF, this mandate can’t be achieved – the metrics are inadequate, the institution-level judgement too blunt. In the most optimistic of interpretations, these issues can be ironed-out as the exercise matures with developing assurance that the measures of ‘excellence’ are meaningful ones. The TEF could mean better information for students, and could lead to improvement in teaching as a result.
With the policy road to year four – and beyond – a bumpy one, with an unpopular and expensive exercise, a more pessimistic view would doubt whether there can be any long-term future for the TEF. The exercise should not be kicked into the long grass before it has had the chance to be tested, and tested at the disciplinary level where it has the most potential. If the goal of teaching enhancement is to be achieved, then there will need to be a sustained commitment to seeing TEF through to maturity in the face of many obstacles.
And if you’ve got this far and still aren’t sure how TEF will work, take a look at Wonkhe’s analysis of what will happen to tuition fees or see the visual guide to the process.
Surely cost will be a significant obstacle to TEF reaching Year Four? Any effort to make the data more robust or useful will increase both complexity and cost, and once we get into discipline level ratings, the aspiration to keep TEF light-touch surely goes out of the window. As a sector we may swallow that, but it’s dishonest for policy-makers to pretend that this isn’t going to generate additional costs for students in the long-term.