Christmas is coming, and, soon after that, the TEF deadline. All over the country, planning departments are working on their TEF provider statements with as much care as goes into the Christmas shopping. As TEF panel chair, over the last few weeks, my email and phone have been packed with queries and questions about the TEF specification and processes.
I cannot now enter a room without being questioned about the TEF. And not just professionally: one of my daughters is a sabbatical student union officer, with lots of things she wants to ask, or tell me. So time for a slightly different approach: from the questions and queries, it’s time to lay to rest some myths.
1. The TEF will punish universities for widening participation
It is wrong to claim that universities which have widened participation will have less good outcome performance than they would otherwise have. It’s not a line I have much sympathy with. Our task as universities is to use our teaching and the wider experiences we offer to widen opportunities and to prepare students for the world they are entering. But even as a matter of methodology, the claim is not true. The TEF metrics are benchmarked, drawing on prior attainment and POLAR quintiles and the questions asked about universities’ metrics data are asked on the basis of benchmarked performance.
2. The TEF is only about metrics; nothing else matters
This is wrong. If it were correct, the TEF would not need a panel and the panel would not need a chair: the TEF awards could be run off from spreadsheets. The metrics give us important information, but we know that they also have limitations, for example in circumstances where a provider contributes heavily to its own benchmarks. In fact, the metrics simply provide starting points for hypotheses about an institution, hypotheses which will be tested against the provider statement as well as more detailed examination of the metrics and metric splits.
The provider statement is a critical part of the process, and every statement will be looked at and considered. Just as in the REF, the panel, which draws on exceptional expertise from right across the sector and beyond, has been assembled on the basis of its experience to make expert judgements. The panel will be able to draw on the full range of information available to it – information that includes regional employment rates and other contextual data, the absolute and benchmarked metric results and the breadth of evidence in the provider submission – to form a nuanced picture of the provider and inform a holistic judgement.
3. The provider statement is simply about ‘explaining away’ metrics
The provider statement is an opportunity to set metric performance in the institutional strategies and practices, but it is also an opportunity to showcase what is distinctive and special about an institution’s approach. It can be celebratory as well as mitigatory, and I suspect that in tone, provider statements that are simply mitigatory will be less impressive than those which give a sense of the relationships between institutional strategies, practices and outcomes.
4. The TEF is biased against certain types of institution; outcomes are pre-ordained
This is a myth propagated from so many different parts of the sector that not everyone who has articulated it can be right: it cannot simultaneously be biased against more and less selective institutions, teaching-intensive and research-intensive institutions, comprehensive and specialist institutions. So it is relatively easy for me to say that it is not true. The panel is drawn from across the sector; the methodology is about outcomes not processes; the provider statement is an opportunity to explore distinctiveness. The job of the TEF, in this instance like the REF, is to recognise excellence wherever it is found.
5. Student views do not count
The TEF specification is clear that institutions should draw, wherever they can, on student perspectives and explain how student engagement feeds the institution’s approach to developing high levels of quality. It might be possible to produce a TEF provider statement without referring to student engagement; it would probably not be sensible. Furthermore, no provider will face a disadvantage in the event of a student boycott.
Almost everyone (re-read that first paragraph carefully) ends their conversations with me about the TEF by saying that they are supporters of the TEF and want to see it succeed in driving a culture which values teaching. Perhaps they are being polite, but I think they are articulating something which many of us have felt for some time: that the performance metrics we have about the higher education sector have become skewed away from teaching for some time. If nothing else, the TEF is an opportunity to re-assert the centrality and importance of teaching to universities.
All of Wonkhe’s TEF coverage so far can be found under #TEF.
6 responses to “Busting five common myths about the TEF”
A welcome set of clarifications. I am particularly pleased that Chris makes clear the role that the contextual statement plays which has been the topic of heated discussion for some weeks now. On the metrics themselves, they may be addressing the skew away from teaching but there is a lot further to go. The challenge as TEF evolves is to find better, more nuanced metrics that reflect the actual contribution of teaching.
They are being polite or you’re only having conversations with a certain group of people. Possibly because those with real criticisms know that the TEF is irredeemable, will be deeply damaging, and see little value trying to tinker with the TEF. You hardly scrape the surface of TEF criticisms here. If the sector wasn’t so scared and feckless the TEF would be in trouble.
I had some interesting discussions at the AUA annual lecture last night around the law of averages. Consensus seemed to be that it seems unlikely that an institution awarded gold or silver will reapply in the subsequent year or two. But could it transpire that if those awarded bronze see an improvement in their metrics and reapply in 2018, we could end up with a silver TEF baseline?
Just five myths? I found ten more – and even one truth! https://academicirregularities.wordpress.com/2016/12/02/ten-myths-and-a-truth-from-the-tef-reading-the-white-paper/
Still yet to address the severe lack of student involvement, consultation or communication in the whole process.