Non-completion and HESES20

What are providers themselves feeling about student non-completion during 2020-21?

It’s a surprise that there is data on this at all, but manipulation of the latest OfS data gives us an unexpectedly candid look at what appears to be going on.

Dr Dan Streetmentioner

Please excuse the confusion of tenses in this article – the HESES data I am playing with refers to data submitted on either 12 November or 10 December 2020, and signed off on 2 February 2021. So the question could perhaps better be phrased as “how did providers reckon non-completion will play out in 2021, based on data as of late 2020 as amended before 2 February”.

The reason we can see an answer is the inclusion of two columns in the HESES20 return – Column 3 asks for a “forecast of years not completed after census date of return” and Column 4 asks for estimated countable years complete for academic year 2020-21”. Column 3 plus Column 4 represents the total number of students that could complete that year, so Column 3 as a proportion of Column 3 plus Column 4 is my non-continuation forecast proxy.

Non-completion (roughly whether a student has completed the modules they intended to study in a year – whether they took the final assessment rather than whether they passed or progressed to the following year) is not the same as non-continuation. There may be reasons other than a student leaving a course for non-completion to occur.

Super-forecasters?

You’re probably thinking at this point that an estimate is very likely to be based on historic data. After all, HESES demands that these forecasts are defensible, and OfS even has the right to wag the finger if you are deemed to be out of whack.

This all changed for 2020-21. Paragraph 15 of Annex D of the guidance cautions:

Providers should ensure that the historical data used to make their estimates of non-completion is not skewed by exceptional circumstances such as industrial action affecting exams. However, providers should ensure that the completion status of students on the appropriate student record reflects their actual completion status. As the coronavirus pandemic is such an exceptional circumstance, we do not expect providers to use data on non-completions in 2019-20 in making estimates for HESES.

As things turned out, 2019-20 non-completions might actually have been the best proxy for 2020-21. As the guidance was published on 9 October 2020 (right in the midst of those awful student residence outbreaks) this was perhaps a forecast that the regulator should have made.

But the practical impact of this is that providers needed to come up with a new, robust method of forecasting non-completion and by the time November rolled around it would have been apparent that a basis on what was being observed last year plus an understanding of the behaviour of characteristic groups from “normal years” would give you a plausible number.

The data

For fundable home students (by default looking at undergraduate full-time or sandwich students) here’s what this looks like:

[Full screen]

A word on non-completion – a number of people make the link between non-continuation (not the same thing as non-completion, an underdiscussed measure) and “poor quality teaching” and these people really should know better. Non-continuation is – in a normal year – predicted by student’s background disadvantage, by ethnicity (which also predicts student’s background disadvantage), by subject area, and by prior attainment – both of which also have strong statistical links to background disadvantage.

It’s fair to see non-completion following similar patterns. For instance numbers of disadvantaged students at a provider in 2020-21 (not, alas, data we yet have) would most likely provide a correlation for these figures. What we know of the impacts of the pandemic suggests that student background disadvantage played a part in predicting how difficult remote study – for example – would be for a student.

The big question we can’t answer is how this compares to a normal year (given that we are used to looking at non-continuation for all students rather than just fundable ones) – we can’t do this because of changes in the way HESES is presented.