Are university admissions racially biased?

A couple of years ago, UCAS took a substantial step forward in opening the admissions debate by releasing the rather un-sexily titled “Undergraduate reports by sex, area background, and ethnic group”.

In my previous life as Wonkhe’s resident data-digger we managed to publish some of the most comprehensive analysis of that dataset. We were able to demonstrate the continued substantial variance in university entry by both ethnicity and social class and, more importantly, point to where the data suggested that there might be bias operating in admissions.

I say “suggested”, because the data provided by UCAS is by no means conclusive proof of bias.

The data provided by UCAS shows where different demographic groups are receiving statistically fewer offers than equivalent applicants with the same entry grades and applying for the same subjects. A full explanation of the data and what it means can be found at the foot of this article.

In the two years since that initial release, UCAS has made two further releases of the relevant data, incorporating the last two application cycles of 2016 and 2017. We now have eight years’ worth of data, from 2010 to 2017, to understand how university admissions for UK full-time, 18 year old, undergraduate entrants has changed for those of different ethnicities.

Do ethnic minority applicants get fewer offers than they should?

Analysis of the 2017 data shows 54 UK universities where applicants of either Black, Asian or Mixed ethnicity were less likely to receive offers than applicants of other ethnicities with the same grades and for same subject. This includes a wide range of universities from across different regions, mission groups, and with widely rates of racial diversity in their entrants. 15 are in the “Sutton Trust 30”, a common proxy for the most “selective” universities in the country. Oxford and Cambridge are not included in the list, though we’ll take a closer look at these two institutions down the line.

Table 1

Table of institutions with possible bias v. Asian applicants in 2017

[table id=152 /]

Table 2

Table of institutions with possible bias v. Black applicants in 2017

[table id=153 /]

Table 3

Table of institutions with possible bias v. Mixed/Other applicants in 2017

[table id=154 /]

Conversely, we find that there are 15 universities where white applicants were more likely to receive offers than applicants of other ethnicities with the same grades and for same subject. Again, the range of universities here is quite broad, including four in the Russell Group.

Table 4

Table of institutions with possible bias in favour of white applicants in 2017

[table id=155 /]

We also find a small number of institutions with possible instances of affirmative offer-making for non-white ethnicities, or perhaps just quirks of the data. With the exceptions of Asian applicants to the University of Bath, and mixed ethnicity applicants to Durham University, all involve quite small numbers of applicants.

Table 5

Table of institutions with possible bias in favour of non-white applicants in 2017

[table id=156 /]

Looking further back than 2017 cycle, my analysis covering the 2013, 2014 and 2015 entry cycles shows very similar results. While many universities drift in and out of the realm of statistical significance in their differences in offer making to difference racial demographics, the aggregate number of universities suggesting possible bias in their offer making remains relatively constant.

Does this mean university admissions are racist?

It might. The data suggests that there must be other reasons why Asian, Black and mixed ethnicity applicants are not getting the same number of offers as “average” students with equivalent UCAS tariff scores and applying for the same subjects. The question the data cannot answer is whether these are factors that might reasonably affect admissions decisions, or whether it is simple unfairness.

In a well-argued response to our 2016 piece, Nottingham Trent University argued that their own analysis showed that Black and Asian applicants with sufficient overall tariff scores were less likely to receive offers because they were more likely not to hold specific A-levels as required by different courses entry criteria. NTU argued that this explained much of the gap in offer rates, and noted that they would take this into consideration in future publication of entry criteria and in providing advice to schools, colleges and applicants on level 3 qualification choices.

NTU’s explanation might apply to several of the universities listed above. If true, it suggests that Black and Asian applicants to university are at the very least at risk of being disadvantaged by their level 3 qualification choices, which may be less likely to meet universities’ requirements than their white peers. Quite why this might be the case is difficult to fathom.

Even if this doesn’t amount to “bias” in the conventional sense, it should nonetheless be of significant concern to universities, schools, colleges, and anyone else responsible for providing information, advice and guidance on higher education choices.

If the theory presented by NTU does not hold up after further analysis in other universities, then it is difficult to look past the explanation of racial bias in offer making, in spite of the caveats listed below. It is hard to deduce, at least at a superficial level, what unites the institutions presented in the lists above that might reasonable explain such differences in offer making across all of them.

To be continued

It is important once again to stress that evidence of this can be found in universities which admit huge numbers of non-white students, and not just in some universities traditionally considered “elite” or “selective”. It shouldn’t come as a surprise, but bias and unfairness appear prone to persist even in relatively diverse university environments.

In future, we’ll zoom in on the issues of fairness in admissions presented by this data to look at the Sutton Trust 30, and later Oxbridge in particular, to understand whether the arguments made both against and in defence of these universities hold up.

Method and caveats

Even with the wealth of data made available about university admissions is difficult to conclusively prove bias against any particular group. Simply looking at the offer-rate – the percentage of a group of applicants made an offer by a university – is insufficient, as it tells does not let us discern between differences in the entry grades of different groups of applicants. It also tells us nothing about the subject which applicants are applying to, as different subjects within universities tend to have very different entry criteria, patterns of offer-making, and demographics of applicants.

The key statistic provided by UCAS to help us discern possible bias in admissions is called the “percentage point difference between the [x demographic] offer rate and the average offer rate”. This statistic controls for the issues noted above – the different entry tariffs of applicants, and the subjects they are applying for – in order to tell us whether a particular group of applicants is still more or less likely to receive an offer than others.

As UCAS put it, “a difference simply means that the offer rate is higher or lower than it is for all applicants who are similar in terms of the subject applied for and a summary measure of their predicted grades”.

UCAS provides a table to identify when this figure is of sufficient statistical significance. In cases where a demographic group of applicants is very small (as is the case with the number of Black applicants to quite a few universities), this makes it difficult to ascertain for sure whether a superficially large gap in offer rates (sometimes as large as five percentage points) is statistically significant enough to be meaningful.

Furthermore, how the controlling factors are taken into account is the source of disagreement between UCAS and other researchers. There are different ways of controlling for factors such as entry grades and subjects of study, and some researchers argue that other factors should also be taken into account. UCAS claim that their way is the most “precise” available. Researchers such as Vikki Boliver at Durham University have been lobbying hard for UCAS to make more underlying data available in order for them to construct their own models which may more may not corroborate with UCAS’s own release.

When a significant difference in offer rates occurs, there is at least enough evidence to question whether it is a result of bias against ethnic minorities within universities. UCAS and universities will argue that there are any number of other factors that might be taken into account. Whether this is a fair defence will likely vary a lot between institutions.

Caution must be urged, but the sector is rapidly running out of ways to easily explain away wide differences in offer rates.

With thanks to David Kernohan for support in data preparation.