Ethnicity in the TEF – what we know (and don’t know)

With the TEF results out and awards now on proud display on websites across the sector, there are still plenty of stories that lie unexplored within the the data. This is one of them.

One of the more interesting yet overlooked features of TEF has been its use of and publication of the ‘split metrics’ – analysis of the exercise’s core metrics divided by a range of student characteristics. We’ve taken a look at at the ethnicity split metrics to see what lessons we can learn.

Give me an R! Give me an N! Give me an SUP! What have you got? A huge caveat

Across the six core metrics there a large amount of suppressed data on BME students. The employment metrics in particular had a great deal of data for BME students marked unavailable. This is particularly evident for FECs and APs, who generally have too few students for such data to be published. Many HEIs have missing data too, predominantly in arts and land-based universities. Jess Moody has already pointed out that at least 10% of institutions awarded a gold TEF had insufficient split data available for BME students. This makes drawing conclusions from the available data challenging.

The core metric for ethnicity split used by the TEF assessors was between white and BME students, but within the contextual data published, a disaggregated BME breakdown was available for our perusal (covering black, Asian, and ‘other’). But this dataset has further publication suppressions due to limited numbers. Consequently, we have only analysed the higher-level data comparing BME and white students, which prevents a truly nuanced analysis.

To determine where differing student experiences by ethnicity appear to have have been significant, we’ve looked at how BME and white students’ flags diverge from each other, i.e. when BME students have negative flags whilst white students have benchmark (=) or positive flags (+,++) and vice versa.

White doesn’t necessarily mean advantaged, but intersectionality is hard to determine

Considering ethnicity in higher education,  it can be easy to assume that all white students will have an advantage compared to BME students. However this is not the case – there are many instances where metrics appear to show more negative flags for white students compared to BME students, particularly at further education colleges. For example, the non-continuation metric had negative flags for white students at 31 institutions, compared to 17 for BME students. Similarly, for the highly skilled employment metric there were 10 institutions where BME flagged negatively, but 22 institutions where white students flagged negatively.

Although this is partly due to the limited data available for BME students, this raises more questions about those white students. Without the ability to look at correlation between POLAR splits, gender, age and ethnicity it is difficult to pull a clear conclusion from this. Essentially, we can’t safely attribute differences to performance in one single group.

BME students do worst at HEIs

Higher education universities were by far the most likely TEF participants to have negative split metrics for their BME students. In nearly all the metrics, with the exception of non-continuation, higher education institutions were the only ones where BME scored negative flags whilst white students met the benchmark or above. The University of Manchester had three metrics where BME students scored negatively compared to white students; employment/further study, teaching on my course and academic support.

Graduate outcomes are the biggest challenge for BME equality

The only metric where there are more institutions where BME students are flagged negatively whilst white students meet benchmark or flag positively is for employment or further study. Our initial hypothesis was that BME students aren’t progressing onto postgraduate study, but the analysis of HESA student enrolment data returns shows no significant difference between the ethnicity of undergraduates and postgraduates. However, we do already know that BME graduates still face significant discrimination in the labour market.  

The divide between white and BME flags isn’t huge

Unlike some of the other metrics, it is rare for one group of students to have negative flags whilst the other scores positively. Instead, we see BME or white students falling below a benchmark (=). In fact, there are only nine instances that show a large divide between the flags:

Can we have some more (data)?

There is no particular regional or contextual data pattern that explains why these institutions have particular issues with their ethnicity split metrics.

Analysing these metrics is challenging due to the many limitations, and raises more questions than they answer. Furthermore, existing BME challenges the sector faces such as the attainment gap and labour market discrimination aren’t necessarily addressed in the TEF metrics. As we move toward the government’s “lessons learned” exercise planned for 2019, the methodological challenges for properly understanding inequitable outcomes in TEF will need to be carefully considered. Such challenges will only be greater when it comes to subject level TEF.