The Sutton Trust on deprivation measures

As new OfS chair James Wharton will tell vice chancellors at a Universities UK event:

Everyone with the talent to benefit from a degree should have the opportunity to get to university, whatever their background.

But what do we know about where students come from? If we wanted to address the needs of applicants that had come from a disadvantaged background, where would we find them?

A new report from the Sutton Trust asks what is, on the face of it, a very technical question – what data exists that can help providers target contextual admissions or other measures? The report suggests that there are a bunch of data sources that purport to offer an answer:

• Years of free school meals eligibility is considered a gold standard measure of childhood poverty – but verified data is not currently available to providers. Eligibility at any time is a decent but surprisingly poor indicator of poverty.
• POLAR and TUNDRA are measures of historic university participation within small statistical areas. Neither is designed to measure poverty, and unsurprisingly neither have a good correlation with low income.
• ACORN is a commercial indicator available for postcodes. It is reasonably good at identifying poverty, but we don’t know what measures or methodology it is based on.
• IMD is another statistical area-based measure, but is not consistent across the UK. The pool of seven indicators involved (income, employment, health, education, crime, housing, environment) under-represent Black and ethnic minority poverty. There’s a specific subscale purporting to show deprivation as it affects children (the IDACI) that forms a part of UCAS’ Multiple Equality measure.
• Output Area Classifications (OAC) are yet another small area measure wherein the ONS puts statistical areas into descriptive categories rather than assigns ordinal ratings.
• The IFS has a composite indicator of socio-economic status of similar predictive power – drawing on privileged access to government administrative data. It’s not really usable outside of what the IFS has already done.
• Family single year income is also not currently available to universities, and is a good measure of poverty though there is statistical noise inherent in using a single year of data
• Parental education can be used as a widening participation criteria – but it is difficult to independently verify an applicants status as “first in family”

In other words, the best ways of actually identifying child poverty work at an individual level – and data is usually self-reported and thus of indeterminate quality. The various area based measures only accurately identify child poverty around half of the time, though some are better than others.

One of the issues here is that poverty, per se, may not be the basis on which you want to make access or participation interventions. There may be under-represented groups particular to a provider or subject area based on other characteristics – ethnicity, age, or simple area-based measures may be better at identifying where you might want to focus attention. Sometimes the much-derided POLAR, used to do what it is designed to do – which is to identify small areas with low historic participation rates – is the best tool to drive equity.

But if you do want to focus on child poverty, it feels reasonable to expect that there should be a reliable measure available to providers. If you are looking at the broad shape of an access campaign, or selecting local schools and colleges to work with, an area based measure is fine – but none of the area based measures is quite good enough to make individual admissions decisions (we’re explicitly encouraged not to use POLAR or TUNDRA for this.

The ideal would be to have individual, independently-verifiable, data on each student – a mode that screams “data protection” – but the Sutton Trust advise that a basket of area metrics would be helpful in identifying students that had experienced child poverty rather than just using one alone. We’re also left with questions as to where we set the cut-off points for each measure (is using the bottom one or two quintiles the right approach?)

The recommendation for the Office for Students is to include a good measure of socio-economic disadvantage (free school meals is suggested) in the data that informs the next round of access and participation plans. I’m honestly uncomfortable with the recommendation of ACORN as the best in class area measure for providers – data in regulatory measures should always be open, buying data can be expensive, and postcodes are not area based in the purest sense.

A commercial measure would also cut across another recommendation – that transparency and consistency should be used in data driven decision making. As we don’t know exactly how ACORN works, that’s difficult to square.