Why has the OfS downgraded graduate outcomes?

The Office for Students (OfS) has reduced the importance of employment outcomes metrics within its quality framework, citing that employment outcomes are influenced by factors beyond universities’ control and that Graduate Outcomes data is imperfect.

But if students are expected to take on significant financial risk when choosing higher education, can regulators justify stepping back from measures of what happens after graduation?

The Ford problem

Ford generally make good cars and in the 1980s and 90s Ford was the market leader.

So, if you were in the market for a new family car in 1990 you could assume that buying the just launched Mark V Escort was a pretty safe purchase. It wasn’t, I drove one, it was a dud.

Prospective buyers quickly worked out they shouldn’t rely on the Ford badge to indicate quality in the case of the Mark V Escort as everyone across the industry panned it. Car nerds at the time like me read about three car magazines a week.

In the end, Ford had to invest heavily in upgrading the Mark V and within two years had significantly improved its quality and sales recovered.

Hidden outcomes

What has this to do with graduate outcomes data and B3 requirements?

Take a look at the employment outcomes data across a range of specific courses on the Discover Uni website and you may be surprised just how often this message pops up:

There was not enough data to publish information specifically for this course. This may be because the course size is too small or not enough students responded to the survey. This does not reflect on the quality of the course.

In cases like these prospective students have to rely on summary data that represents a much broader range of courses, and of course the marketing materials designed to promote the course.

The OfS has published the outcomes of their consultation on their future approach to quality regulation. Part of the review focused on what is called the progression indicator in Condition B3. For those that aren’t that interested in the minutia of university regulation, this is the part of the regulatory framework that is supposed to ensure that the sector is delivering for students.

Here is a reminder of what the OfS is supposed deliver on, note the reference to careers, progress and value for money:

We aim to ensure that every student, whatever their background, has a fulfilling experience of higher education that enriches their lives and careers.

Our four regulatory objectives. All students, from all backgrounds, and with the ability and desire to undertake higher education:

• are supported to access, succeed in, and progress from, higher education
• receive a high quality academic experience, and their interests are protected while they study or in the event of provider, campus or course closure
• are able to progress into employment or further study, and their qualifications hold their value over time
• receive value for money.

The overall report is 89 pages long, but for simplicity I’ve pulled out two paragraphs that summarise the OfS’s position: “There were high levels of support for the removal of the requirement for providers to meet a minimum threshold for progression, generally because respondents agreed that the factors affecting employment outcomes are complex, and that these are less directly under the control of the provider than whether a student continues or completes their course” and “Some of those who supported its removal also commented on perceived weaknesses in the Graduate Outcomes Survey (GOS) data, such as the response rate achieved…”

The OfS approach

It is no surprise that many institutions would rather not be held to account for graduate employment outcomes.

Universities control their marketing campaigns, admissions processes, course delivery and marking structures, but career outcomes are much harder to influence. Influencing employment outcomes is a complex undertaking and does require investment over time.

The Graduate Outcomes survey itself has been managed in a way that has seen response rates fall from 52 per cent to 32 per cent over the last four years (The Destinations of Leavers from Higher Education was imperfect, but when universities managed this themselves response rates were over 80 per cent).

Instead of pushing to increase response rates and thereby the data available to students, the OfS is using the low response rates as a reason to remove career outcomes as a performance measure.

Fix the data, not the metric

The fact that the UK has a loose link between subject studied and career options compounds the need for strong outcomes data – Institute of Student Employers (ISE) data shows that over 80 per cent of employers do not specify subject studied when recruiting.

I’m going to pick on law as it is relatively easy to get the stats, but the same applies to the majority of disciplines. In 2022 there were 5,953 training contracts or barrister pupillages available with employers, yet 26,290 students were accepted onto law degree courses.

These numbers become more dramatic when you consider that many law firms recruit students who have not studied law. This is not to say “don’t study law,” but prospective students should be made aware of the implications of their choices.

Diluting B3 measures may seem an attractive option to the sector. But if we are going to have a system of uncapped course numbers where students’ choice is the dominating factor, then they should have the information and consumer rights resources to inform and protect them.

The B3 conditions are conditions for registration, they are not trivial. By watering down the very performance measures designed to enhance student outcomes, the Office for Students is not acting in students’ interests, which in the long run will only harm the reputation of the sector.