I’ve always wanted good data what the impact of entering university via clearing would be on non-continuation.
A new report from the Office for Students finally lets me have it – and there’s around a one percentage point difference in continuation rates between students applying with A levels placed via a conditional offer and those placed via “other UCAS routes” (a category which includes primarily clearing, but also adjustment, UCAS Extra, and very late confirmation of offers). A statistically significant difference.
There is a similar but slightly smaller effect, also very much on the bounds of statistical significance, for students holding an unconditional offer. You probably won’t be surprised to learn that this is the effect that OfS is focused on, and that gives the report (“Data analysis of unconditional offers” ) its name.
Behind the headlines
But we should worry about both. Unconditional offers have grown remarkably in volume – from 1 per cent of applicants in 2013 holding an offer with an unconditional component, up to 39.5 per cent of applicants (some 91,000 individuals) in 2019. But these startling figures disguise a more mundane reality. Some 24,055 English domiciled 18 year olds were in higher education based on an unconditional offer in 2017-18.
In comparison, the number of applicants entering a course through clearing rose from 57,000 in 2013 to 73,320 in 2019. For English domiciled 18 year olds in 2017-18, the figure was 24,490. So there are two groups of pretty much equal size in English HE that have a continuation rate a percentage point lower than – all things considered – it should be.
Which things considered? There’s a pretty sophisticated model here that looks at all the factors you might expect: provider, entry route, predicted qualifications, subject of study, level of study, disability, sex, ethnicity, local or distance learner, POLAR4, IMD (I’ve plotted the impact on continuation of those aspects here). The odd one out is “predicted qualifications” – why use these rather than actual qualifications?
Well, Annex D of the report tells us that:
UCAS research has shown there to be a negative relationship between unconditional offers and Level 3 attainment. An applicant whose attainment is negatively affected by their unconditional offer might then be less likely to continue in higher education as a result. Therefore, to avoid correcting for this, our [main] model uses predicted entry qualifications (which are unaffected by whether an applicant receives any unconditional offers)”
The angels that dance on the head of a pin
Where this falls down for me is that we assume that an observed “negative relationship” is also a causal relationship. The UCAS model controlled for sex, POLAR4, ethnicity, school type, region, provider, and subject (the latter both of the course that the offer was held to, and A level subjects). It compared the predicted grades for the best three A levels to a binary variable that said whether or not a student had missed their predicted grades by three or more points.
I’m going to assume that the finding is significant (we’re not told explicitly, but it looks it). We don’t get coefficients in the UCAS study, but there’s one statement worth noting:
Other factors that may impact applicant attainment (for example, motivation) are not controlled for in the model”
Now, as I understand it, our speculative hypothesis is that unconditional offers are bad because they make people less motivated to do well at A level. Motivation seems like a pretty basic thing to ask about in a survey of applicant attitudes, and even though there is a report a question asked about feelings while waiting for exam results in 2019 (those with unconditional offers are calmer, less stressed, less worried, and less uncertain) we don’t get any sense of motivation. So we don’t (really) test that hypothesis and other than the suggestion that women are more likely than men to miss their predicted grades we don’t get any other information on other factors that might have a stronger impact.
I asked UCAS about the rest of this survey – I understand it was a much wider survey with multiple angles, and that it has not been published.
This bit is the good bit
Perhaps sensing that this rationale was on slightly shaky ground, OfS also ran an alternative model (“Model 2”) – which used actual grades rather than predicted grades. Looking at the same data, making this one change, there is no longer a statistically significant relationship between unconditional offers and non-continuation.
While you may argue – and OfS does argue – that this backs up the idea that an unconditional offer lowers A level performance by lowering motivation, it isn’t as clear cut as that.
The OfS report says that:
This would be consistent with what would be found if poorer performance at A-level, relative to predicted grades for those placed through unconditional offers, were driving the lower continuation rates of these entrants”
But it stops short of attributing a causal relationship – we still don’t know if an unconditional offer demotivates A level candidates and leads to exam underperformance, or if there is something else significant about that group of candidates that leads to higher rates of non-continuation. It’s the kind of thing that might be worth knowing.
The clincher here is that “Model 2” shows us no significant effect for A level students entering HE via another UCAS route (although effects still appear for those entering with other qualifications). What we appear to have demonstrated is that candidates who do worse than expected at A level also do worse than expected in the first year of an undergraduate degree. There was no unconditional offer to demotivate those unexpectedly entering clearing.
To be clear about clearing
Though we don’t know for certain about why this effect exists, we do know that the effect is at a similar level for two similar sized groups of students – those entering via unconditional offers, and those entering via those “other UCAS routes”. And while the campaign against unconditional offers from government, DfE, and OfS has been long-standing and vociferous, there has been a concerted campaign to nudge more applicants into adjustment, self-release, and clearing.
It’s been a policy goal for some years to convince talented young people from disadvantaged backgrounds to “trade up” to more selective providers – interventions like Clearing Plus exist to help applicants make helpful choices in clearing, but remember what the government said about it:
UCAS is developing a new, personalised Clearing system for students this summer. This includes Clearing Plus, a new service which matches students to universities or other opportunities based on their achievements and course interests. If students’ calculated grades exceed their predicted ones, it can suggest alternative courses with higher entry requirements.”
I mean, this isn’t accurate, but as a policy wish it’s pretty clear what the position is. We got “self-release” last year, and depending on your preferred prediction clearing could be busier than ever for UK domiciled undergraduates this year.
On one hand, government and regulator are using the strongest possible language to convince providers to get rid of (anxiety reducing) unconditional offers, citing a significant linked rise in non-continuation – and on the other hand we are actively encouraging young people into a pressurised higher education marketplace despite a significant linked rise in continuation. And we still don’t know if the hypothesised “lack of motivation” that would lead to worse A level results with an unconditional offer is real.
Great summary of an interesting report. We’ve noted the Clearing link to non-continuation at my university for some time. UCAS, of course, is heavily commercially invested in Clearing. Thankfully, my team is focused on the best advice and guidance we can give each individual at whatever stage in the admissions process they reach out. We need advice and guidance voices to be at least as strong in student recruitment as those who know about, provide or buy (at vast expense) marketing, influencing and micro-targeting of our HE-destined young people.