TEF results – A missed opportunity for progress on equality

ECU has been following the (rapid) development of the TEF in a series of blogs and publications with a particularly keen eye for two main reasons.

Firstly the TEF design stated a desire to ensure the exercise looked at the experience of ‘disadvantaged’ students, and secondly and more specifically, many of the assessment categories broke down data in student ‘types’ across a range of characteristics including some which legally institutions should be paying attention to anyway:

Protected characteristics under the Equality Act 2010

• Age
• Disability
• Ethnicity
• Sex

Other identity-related characteristics

• From 'disadvantaged' areas
• Domicile (UK/EU/other)
• Welsh medium (Wales only)

Course specific

• Year of study
• Mode of study

Following on from some of ECU's previous commentary on the opportunities and challenges of the TEF, with the release of the panel ‘statement of findings’ on each submission, we see a missed opportunity for equality, diversity and inclusion.

Overconfidence

Time and again in the Gold award panel statements we see a conclusion that “that students from all backgrounds achieve consistently outstanding outcomes.” Such a declaration regarding ‘all backgrounds’ and ‘consistency’ needs careful handling.

Firstly the TEF metrics did not set out to provide a lens through which to examine adequately ‘all’ different sorts of students, even by protected characteristic. Such ‘consistencies’ at a highly aggregated level risks homogenising the student experience, and smoothing over the ‘gaps’ and inconsistencies in experiences and outcomes which we know as a sector exist – in terms of belonging, in terms of employment salary, and in terms of progression to PGT and PGR.

The metrics are also limited by small numbers so that the ‘minority’ experience is in some cases effectively removed from the picture.  So where (for example) a university might have so few Black students (or so few responding to the NSS and DLHE) that it doesn’t even generate usable metrics – which is the case for at least 10% of the Gold awarded higher education institutes, it seems a curious state of affairs to indicate with a statement a confidence in consistency of outcome for ‘all students’.  

Of course, the role of the TEF is not to in itself examine underrepresentation per se. But the tensions between diversity, inclusion, underrepresentation and teaching practise, as evidenced in work around the BME attainment gap (for example), indicate that this is a big missing part of the story, and care needs to be taken in communication of these results to the next generation of applicants.

We also need to think about how current students accept this. For those campaigning for a ‘decolonisation’ of the curriculum, or facing transphobia, Islamophobia, or anti-Semitism on campuses, is it heartening to be told that students from all backgrounds – or even most backgrounds - are thriving? Does this indicate trust and understanding and that your voice is being heard in the sector? This is one reason for many students' union's opposition to TEF, which is more than just about fees, and will be an important space for discussion in the future,

Forgetting equality

In reviewing the TEF outcomes and submissions what is most worrying is often what is not said. At a time when any national review of university teaching quality could be prioritising the urgent challenges of BME attainment gaps, and exploring the impact of increasing mental health issues on learners, our initial review of provider submissions finds that few make any meaningful mentions of race and unseen disability.

This is not to forget that some fantastic work is going on in the sector. There are hundreds of committed staff and students building momentum to transform their learning and teaching spaces into inclusive places where all can thrive. Indeed, our work with the Race Equality Charter (for example) sees a huge step-change in many institutions, and it was heartening to see the influence of this in some submissions.

But sadly, this work seems to still be on the periphery of the TEF conversation, subsumed into euphemisms and generalisations about ‘disadvantage’ and ‘types’ of students. In fairness, Chris Husbands has been clear that “no one has ever claimed that these are the only things worth measuring about higher education”. But silence can have damaging effects, and indeed, we have heard directly from some institutions that 'we don’t have any negative flags for BME for TEF, so we don’t really need to do anything about race'.

All-in-one

TEF has perhaps suffered from a similar problem to the institutions that it is reviewing. Thought there is a commitment to improving equality and trying to understand the varied identities of students, it fails to get prioritised. It is too difficult to confidently interrogate minority students' experiences when they are more challenging to show on a spreadsheet. Some more detailed statements of panel findings which critically examine the split metrics in more depth would certainly be more helpful for the future.

Husbands has acknowledged that “there is some distance to go in some institutions to understand and address disparities in performance”, and we know these are early days.

We at ECU commend the hard work that has gone into TEF in such a short time frame, and we also know that in some institutions TEF has been a real incentive to do more meaningful examinations of differential outcomes, particularly for BME and disabled students. Some foundation for better equality work is there, and we wish to work with the lessons-learned exercise to strengthen the TEF into a more meaningful tool for equality.

TEF should not only examine if teaching excellence is delivered to all groups of students but instead demand that a fully inclusive experience is part of the definition of 'excellence'.

Going forward, ECU has the following recommendations for the new Office for Students and future TEF panels:

• Flagging differentials between characteristics (for example, various ethnicities, ages, genders).
• Prioritising the degree attainment gap and its links to teaching and learning.
• Disaggregating ethnicity and disability into more useful categories.
• A timeline for future consideration of different protected and ‘widening participation’ characteristics.
• Consideration of ways to examine minority experiences where representation is numerically too low to include in metrics – with probing questions as to whether that representation is unusually low for the institutional context.
• Providing a summary overview of learnings of equality and diversity teaching ‘excellence’ from the submissions.