Broad OfS data on a subtle subject
David Kernohan is Deputy Editor of Wonkhe
Tags
Put together for the Westminster government’s Commission on Race and Ethnic Disparities we get high level subject (STEM or not-STEM) and provider group (tariff groups) splits on continuation and non-continuation rates for the four academic years up to 2017-18. If you are thinking about the content of the recent letter to the equalities minister from the Commission, and how it said that:
the ‘grouping’ of all ‘Black’ pupils together tells us very little specifically, and risks damning a child’s achievements and future prospects
You may conclude that the use of top level ethnicity characteristics alongside these other broad categories runs the risk of not supporting the Commissions stated aim to report:
based on a more granular approach to data
What makes this doubly odd is that OfS’ own “access and participation data dashboard” offers a more nuanced view of providers (and there’s additional data on subject of study), and also adds an extra year to the time series, and (very helpfully) also includes confidence intervals. Similarly, TEF data would offer a lot more detail.
Now, there may be very good statistical reasons for providing data in such broad strokes – but we are not presented with them. As the Commission labours the point that we need more nuanced data on this complex topic to make sense of what is going on, it is at least a reasonable question to ask.
Findings
Here’s a quick plot of the continuation data.
There’s nothing here that would surprise anyone familiar with existing data on this topic – OfS notes that:
The gap in continuation rates between white and black students is wider in non-higher tariff
providers (6.1 percentage points) than in higher tariff providers (1.8 percentage points)
But this prompts more questions. Are there proportionally more Black students in non-higher tariff providers? (yes there are). Are there other personal characteristics among that population linked with lower continuation rates? (yes there are). This is exactly the point the commission is making.
A note on tariffs
I’m not a fan of categorising providers by entry tariff. Average entry points can be skewed based on subject mix as well as by institutional mission and locality. And the ranking (“high tariff” this the top third of non-specialist providers) is going to change every year – OfS takes an average of three academic years (good) but these start in 2012 (questionable).
The problem is we all assume we know which providers are “high tariff”, and this can lead us to make inappropriate assumptions and generalisations. If you are going to use provider groupings in public data, it should really be clear which providers are in which group. I’ve asked about tariff rates and groups for providers before to be told they are “commercial in confidence”, but UCAS published a list yesterday of 18 “selective high tariff providers” (Annex A) and the world appeared not to end.
Thanks you for taking the time to look at our statistics, we agree about the need to always look behind the top level groups which is why my colleague Chris Millward pulled this out in his commentary which you can find here https://www.officeforstudents.org.uk/news-blog-and-events/press-and-media/new-ofs-analysis-examines-differences-in-access-and-continuation-between-different-groups-of-students/
OfS do publish the list of providers included in our high tarriff grouping you can find it here: https://www.officeforstudents.org.uk/about/measures-of-our-success/participation-performance-measures/gap-in-participation-at-higher-tariff-providers-between-the-most-and-least-represented-groups/ . Obviously it would have helped if we had linked to this in our publication.
There is not more up to date data in the A&P dashboard, the continuation data are from the same year, continuation always lags access as we need to wait for the data for the second year of study to be returned
Hi Richard – thanks for your comment, which as always is very welcome.
I’d love to see finer grained analysis from OfS on this topic in future, and I’d be very grateful for any developments towards a more sophisticated way of comparing providers by tariff.