Understanding differences in student outcomes

It’s interesting how the same piece of policy research can appear to have such different interpretations.

In its last week, HEFCE published Differences in student outcomes: The effect of student characteristics. It is, therefore, one of the last reports published in a long series that will migrate across to the National Web Archive.

Once I was past the pre-nostalgia (were we allowed to be nostalgic for HEFCE before it closed?) I was struck by the way that the data presents some marked achievement gaps. Using a large dataset from across the whole of the English university sector, a series of graphs shows where the gap grows as you compare students with the same entry grades but different characteristics.

However, for Bernard Rivers commenting in THE, this was proof of grade inflation. This hadn’t crossed my mind reading the HEFCE report – of course, there were differences in the 2013/14 and 2016/17 cohorts – but that isn’t its focus. All the same arguments apply: is it grade inflation or an increase in student achievement (set against a better focus on assessment outcomes, the increased importance of the “good degree”, or just better student outcomes).

Bringing these two things together, however, shows the scope for both issues to play out. I’m sure that many universities will have been grappling with the attainment gap, shown in differential student outcomes. I know that Nottingham Trent University (NTU) has. It has committed itself to a long-term strategy of “Success for All”, putting in place a series of actions at both department and university level, to try its best to work to overcome the gaps, and making strides in addressing attainment and employment disparities.

At the national level, the largest identified factor is the entry qualification of students. This pattern is most marked by the gaps shown with the performance of students with BTEC National Diplomas. Students with BTECs have a smaller proportion of good degrees than all students with A-levels (even BTEC students with 3 distinctions get a smaller proportion than A-level students with 3 Cs).  

The report goes on to split the data by student characteristics; age, gender, disability, ethnicity, and educational disadvantage. Annex F sets out the impact of all the characteristics, which allows them to create a model of expected differences; no student has only one of these characteristics, they work together.

Gender differences

As an example, here is the chart showing the difference by gender. There is an issue with the under-performance of male students, but the lines show how that difference grows to its largest extent with students with B grades at A-level.

Source: HEFCE

Ethnicity

Similarly, the chart about ethnicity shows a marked gap between the four groupings used in the data, with a difference of 17 percentage points between white and black students. The gap opens out to alarming levels when looking at those with BTECs.

Source: HEFCE

This is where the intersection of characteristics comes to the fore. At NTU we have a specific strand of work looking at actively supporting students who have previously taken a BTEC. Our departments have BTEC champions, working as a group, who support colleagues in creating an accessible environment for students. We focus on pre-arrival and induction measures and have targeted support sessions (particularly around assessment as examinations will be less familiar). We have reviewed our curricula to ensure linkages are there. We know that many universities are also active in this area, and there is a lot of sharing of good practice across the sector. As we support students with BTECs, we will know that will also support a number of traditionally disadvantaged groups, not least BME students. We’re not alone; our hosted symposia have attracted staff working on the same issue from many universities.

In turn, the HEFCE report looks at employment outcomes. There are gaps here, and there isn’t a university employment or careers service that isn’t running initiatives to help overcome these – although we know that opportunities in the workplace are not going to be evenly distributed.

Action required

Throughout HEFCE’s publication record each report or circular has included instructions on what a HEI needs to do with the document. HEFCE 2018/05 signs off as follows:

Action required

19. This document is for information only.

Which is true, but what’s interesting now is what providers continue to do with this information. The good news is that OfS has taken forward the ownership of this data, publishing a blog by Rebecca Finlayson highlighting it, and by Chris Millward for the Guardian’s HE network. This will be an interesting test of how OfS uses its levers to affect change. When stating what OfS will do, Millward writes that they will apply pressure to universities. Meanwhile, OfS has trimmed £30m from the premium funding to support successful student outcomes for full-time undergraduates, as HEFCE did the previous year.

The onus for action lies firmly with universities and the good news is that many of them are already very active in this area. We’ve already understood that access doesn’t stop at admission, and we must work on attainment and student success. Potentially that’s bad news for Bernard Rivers and others who worry about grade inflation, we are going to work as hard as we can to close those gaps – supporting our students, aiming for success for all.  

4 responses to “Understanding differences in student outcomes

  1. I’m not surprised that students with A levels and BTEC perform differently. At schools the students who do BTEC are the less academic, who would struggle with A levels, so it can’t be surprising that they would require extra support at University. They are not equivalent qualifications, and treating them as such benefits nobody. They are for different cohorts. I’m also interested how anyone could say there has not been grade inflation. The number of students getting 1st class degrees has risen dramatically, and people are not getting cleverer to the same extent. It has happened in schools too. Could we go back to having flexible grade bands whereby a certain % of students got the top grade each year? Then we could really spot the high achievers and they could get the credit they deserve rather than being one of the growing mundane cohort with 1st class degrees.

  2. Grouping the IB students into one point can’t be helpful here. The highest and lowest achievers will likely have very different degree classification outcomes.

  3. Chinese and Indian students are often high achievers in terms of GCSE and A level. Their average earning in LEO released in March seems higher than the students from other ethnic groups too. It would be helpful to see their degree classification and explore the link between culture, academic success and career outcome.

Leave a Reply