The UK higher education sector comprises a diverse group of institutions with variable entry requirements and variable methods for calculating students’ final degree classifications, yet degrees are theoretically equivalent.
How can we make such claims?
Degree classification algorithms (DCAs) are a formula that is applied to eligible module-level results to calculate the final degree grade and classification – that is, a ‘final’ mark and whether the student graduates with a first class classification, upper second, lower second, third, or fail.
Wonkhe has previously covered the variation in DCA across the sector. This includes different approaches to the weighting of levels of study, discounting of low grades, and the inclusion of first-year studies. The bespoke nature of regulations at each institution means that policies related to resits and borderline thresholds also contribute to variation.
What does this mean for students?
Research clearly shows that where DCAs are applied some students will be awarded a different degree outcome (up to a full grade boundary), depending on the rules of their awarding institution. This has implications for equity across institutions. Don’t forget we argue that the FHEQ levels and external examining process establish equivalence across disparate institutions. There would appear to be a disconnect in the logic where different institutions then apply different DCAs to students’ module-level results.
Frequently terms such as exit velocity are invoked in discussions related to DCA construction but there is little empirical evidence presented, to satisfy the question of whether exit velocity is uniform across the sector or whether it is used to adjust student outcomes such that there is greater institutional equivalence. This argument is given some credence by survey findings that 38.7 per cent of respondents reported that they had altered their algorithms in the preceding five years “to ensure students are not disadvantaged vis-à-vis their peers at similar institutions”.
We wonder how aware students are of these differences across the sector and the impact that they have on their behaviour as they adapt to enhance their expected degree classification.
Re-ranking
Much of the focus has been on the degree outcome which takes for granted the process of combining student marks into the final classification. It is time to start to question the extent to which DCAs contribute to grade inflation by examining the underlying modular outcomes. In extreme cases, would the Office for Students start to question the credibility of some degrees under the B4 condition?
We can envisage that awarding gaps can be expanded or narrowed in some circumstances through the application of different degree algorithms that effectively re-rank students. This means that a two-step approach to awarding gaps is required, firstly addressing the module level input marks and secondly addressing the DCA effect.
Grade point average (GPA), whilst common internationally, has not been adopted in the UK despite attempts by the Higher Education Academy back in 2015. Even where it is deployed variation in the calculation exists so it may not address some of the concerns raised whilst generating significant uncertainty through any change process.
What next?
In our view, no magic wand will address the existing limitations of DCAs, and we suggest that a coherent approach in the UK has to start by gathering more data related to issues at the sector, institution, and student levels. This would enable more effective decision-making around the routes forward and the continuum of available approaches, from the radical option of adopting a single algorithm across all institutions, to the (perhaps more reasonable) reduction in variance, dovetailing with the UK standing committee on quality assessment’s recommendation to trim the number of UK DCAs to four.
Our recent article ‘Opening the black box of degree classification algorithms: Towards a research agenda’ seeks to frame the discussion around a new research agenda to help answer some of the ongoing questions related to DCAs.
Don’t forget the impact of DCAs on student behaviour and stress. Almost all courses I interact with have first year not counting so that students have the opportunity to “settle in”. Was in an interesting discussion where one academic was saying that first year should count so that each year counts less to the final degree and students will therefore be less stressed in each year as they had “banked” their performance. Another one countered by saying that it should be 100% weighted on the final year as that gives students time to experiment / find out what works best for them and only then does it count. I could see both views and suspect different students will prefer different approaches. The 3rd option is to give students the best of both worlds to remove stress (best of each year counting or final year) but that will result in inflation.
Good point Andy – as we know the first two approaches to first year grades are in operation across the sector.
To add to this, a trial at Oxford Brookes introducing a GPA (e.g. including first year grades), resulting in a small but significant reduction in disadvantage to equity-seeking groups, and benefits to mature and BAME students…
https://www.researchgate.net/publication/303382037_The_why_what_and_impact_of_GPA_at_Oxford_Brookes_University
I’ve been waiting for a while for someone to ‘do the math’ and set up a league table focusing on where it’s easiest to get a first with a certain set of grades…
It’s a lot more nuanced than just ‘grades’, with some universities (note small u) indulging in interjectionally adjusted and/or potential/actual monies donated awarding. I saw that at my sons graduation where a person who ticked enough boxes/paid enough money got a 1.1 even though they failed to turn work in, didn’t attend lectures or group work sessions. Some of their fellow graduand’s from the same courses comments and disbelief was very revealing… The knock on effect doesn’t adversely effect the beneficiaries of such awarding, but it does effect the rest whose degree is devalued, my sons degree whilst noted by employers is regarded with distain by most as they know ‘his’ university is prone to ‘over awarding’ from their past experience of Graduates from that university. Whilst they won’t admit it, through fear of another ‘blacklisting’ scandal, the H.R. professionals in many organisations pool such knowledge, often reinforcing their own biases from their own University education.
You comment an ever changing variation of this on every degree classification article and it is still nonsense.
I think there is a risk of presenting an overly simplistic picture, as the DCA is only one part of what drives degree class.
From previous research, both the volume of permitted resits and whether or not the resits are capped will have an impact. The former can increase the proportion of students passing with lower awards, rather than withdrawing, which will produce different %s for the proportions of certain degree types. With the latter, uncapped resits produce higher outcomes.
Then there is what happens with marginal/borderline candidates, then there is the fact that, ultimately, marking is variable. There are expectations of consistency within a provider, and across subject disciplines, but ultimately people use slightly different criteria and produce different results. Curricula vary, as do assessment methods; both produce further variation. There will be many other factors (e.g. are trailing fails permitted, how do compensation and condonement work, what is the pass mark).
And, of course, most DCAs are just different (i.e. often some students will do better and others do worse, because of the variation).
From looking at our DCA within the institution, it looks like changes to the DCA don’t necessarily change outcomes significantly, because subject areas ‘correct’ behaviour to return comparable outcomes .
Because there are so many moving parts in determining outcomes, I am not convinced that fixing DCAs in place would create consistency, though they would mean everyone had to review what the impact on their practice would be (and correct as needed). I think the question is whether any common expectations can be agreed (i.e. can some things be ruled out).
Thanks for this, Andy! Appreciate your thoughts. It is definitely a much more complex picture than DCAs alone, and we definitely wouldn’t want to paint a picture that says that this is the only contributing factor… but it certainly is an underexplored one. As you rightly raise, uncapped resits, and other institutional guidelines, are also big issues.
Previous research has pointed to an upward ‘drift’ due to DCA changes though, especially in the years where many institutions started to change their algorithms. From our recent article, for example:
“… in a 2021 survey of HEIs, it was found that 48 (38.7%) had changed their DCA within the last five years ‘to ensure that [it] does not disadvantage students in comparison with students in similar institutions’. This algorithm re-writing exercise indicates some link with grade inflation as ‘over time’ it has ‘create[d] an upward movement in award outcomes […] that is irrespective of changes in student performance’ (UKSCQA 2021a, 7)”
And the issue of cross-institution comparability is a tough one as well… and previous research has suggested that students with identical marks from different universities with different algorithms could end up with drastically different marks (of up to 10%).
Focussing only on DCAs won’t give us the full picture, but adding them into the discussion alongside the variety of other factors, is important, and it’s interesting that it hasn’t featured as much on the radar as it might!