Understanding differences in student outcomes

It’s interesting how the same piece of policy research can appear to have such different interpretations.

In its last week, HEFCE published Differences in student outcomes: The effect of student characteristics. It is, therefore, one of the last reports published in a long series that will migrate across to the National Web Archive.

Once I was past the pre-nostalgia (were we allowed to be nostalgic for HEFCE before it closed?) I was struck by the way that the data presents some marked achievement gaps. Using a large dataset from across the whole of the English university sector, a series of graphs shows where the gap grows as you compare students with the same entry grades but different characteristics.

However, for Bernard Rivers commenting in THE, this was proof of grade inflation. This hadn’t crossed my mind reading the HEFCE report – of course, there were differences in the 2013/14 and 2016/17 cohorts – but that isn’t its focus. All the same arguments apply: is it grade inflation or an increase in student achievement (set against a better focus on assessment outcomes, the increased importance of the “good degree”, or just better student outcomes).

Bringing these two things together, however, shows the scope for both issues to play out. I’m sure that many universities will have been grappling with the attainment gap, shown in differential student outcomes. I know that Nottingham Trent University (NTU) has. It has committed itself to a long-term strategy of “Success for All”, putting in place a series of actions at both department and university level, to try its best to work to overcome the gaps, and making strides in addressing attainment and employment disparities.

At the national level, the largest identified factor is the entry qualification of students. This pattern is most marked by the gaps shown with the performance of students with BTEC National Diplomas. Students with BTECs have a smaller proportion of good degrees than all students with A-levels (even BTEC students with 3 distinctions get a smaller proportion than A-level students with 3 Cs).  

The report goes on to split the data by student characteristics; age, gender, disability, ethnicity, and educational disadvantage. Annex F sets out the impact of all the characteristics, which allows them to create a model of expected differences; no student has only one of these characteristics, they work together.

Gender differences

As an example, here is the chart showing the difference by gender. There is an issue with the under-performance of male students, but the lines show how that difference grows to its largest extent with students with B grades at A-level.

Source: HEFCE

Ethnicity

Similarly, the chart about ethnicity shows a marked gap between the four groupings used in the data, with a difference of 17 percentage points between white and black students. The gap opens out to alarming levels when looking at those with BTECs.

Source: HEFCE

This is where the intersection of characteristics comes to the fore. At NTU we have a specific strand of work looking at actively supporting students who have previously taken a BTEC. Our departments have BTEC champions, working as a group, who support colleagues in creating an accessible environment for students. We focus on pre-arrival and induction measures and have targeted support sessions (particularly around assessment as examinations will be less familiar). We have reviewed our curricula to ensure linkages are there. We know that many universities are also active in this area, and there is a lot of sharing of good practice across the sector. As we support students with BTECs, we will know that will also support a number of traditionally disadvantaged groups, not least BME students. We’re not alone; our hosted symposia have attracted staff working on the same issue from many universities.

In turn, the HEFCE report looks at employment outcomes. There are gaps here, and there isn’t a university employment or careers service that isn’t running initiatives to help overcome these – although we know that opportunities in the workplace are not going to be evenly distributed.

Action required

Throughout HEFCE’s publication record each report or circular has included instructions on what a HEI needs to do with the document. HEFCE 2018/05 signs off as follows:

Action required

19. This document is for information only.

Which is true, but what’s interesting now is what providers continue to do with this information. The good news is that OfS has taken forward the ownership of this data, publishing a blog by Rebecca Finlayson highlighting it, and by Chris Millward for the Guardian’s HE network. This will be an interesting test of how OfS uses its levers to affect change. When stating what OfS will do, Millward writes that they will apply pressure to universities. Meanwhile, OfS has trimmed £30m from the premium funding to support successful student outcomes for full-time undergraduates, as HEFCE did the previous year.

The onus for action lies firmly with universities and the good news is that many of them are already very active in this area. We’ve already understood that access doesn’t stop at admission, and we must work on attainment and student success. Potentially that’s bad news for Bernard Rivers and others who worry about grade inflation, we are going to work as hard as we can to close those gaps – supporting our students, aiming for success for all.