For years it has been an open secret that predicted A level grades are not particularly good predictions of actual exam results.
If you were at Wonkfest in 2019 you’ll probably recall Mark Corver talking about this issue – and I’ve written about the topic the summer before last (noting that the error is equivalent across POLAR quintiles).
In UCAS’ quest to spin the end of cycle data reports out for a full two months we get a detailed look at offer making and the student response to it from the sector admissions body – both further details (after the “insight report” last month) in terms of the currently fashionable “conditional unconditional” offer, and in general terms on the way predictions work in admissions.
Accuracy and utility
University admissions have long been a political issue – Labour’s “longest unsuccessful UCAS personal statement in history” even featured calls to move to a system of post-qualification admissions. With only one in five 18 year olds meeting or exceeding their predicted grades in 2019, there are clearly questions to be asked.
However the margin of error is highly predictable – predictions generally lie within 2-3 points above the actual grades, and this year’s figure is 2.35 points. There are differences based on attainment – higher predicted grades are likely to mean a smaller average difference – and more likelihood that an applicant would meet or exceed predicted grades.
UCAS has already moved to address some of these issues – there’s guidance for schools, and exploration of “advanced modelling based on information about the previous achievement of a student, such as GCSE or National 5 grades and their context, could create a data-driven addition or alternative to predicted grades.”
The emphasis in guidance and reporting is that predicted grades should be seen as one part of a holistic system – a nod to more contextual approaches to admissions playing a wider role. Intriguingly there has been a rise in the acceptance rates for applicants holding three E grades over last year.
And a new statistical model suggests that students holding a conditional unconditional offer as their firm choice are 11.5 percentage points more likely to miss their predicted
grades by three or more grades than if they held a standard offer.
Conditional unconditional love
But it appears that even an unconditional offer is not enough to get applicants to accept a place any more – those in receipt of a conditional unconditional offer are only 1.3 percentage points more likely to accept it as their first choice than any other individual offer
We’re all well aware that a quarter of graduates have received at least one offer with an unconditional component – this makes up 14.2 per cent of all offers. Eight point five per cent of all offers were of the “conditional unconditional” type so often disparaged by ministers. The use of unconditional offers of all types has risen again in 2019, despite the warnings over the summer. This cycle, a record 137,805 offers with an unconditional component were given to 18 year old applicants from the UK.
Primarily direct unconditional offers are made to applicants from lower POLAR quintiles, with conditional unconditional offers more prevalent in quintiles three, four, and five.
There’s also data by subject area – as you may expect the portfolio-heavy world of creative arts and design has the highest proportion of direct unconditional offers, but conditional unconditional offers are more prevalent in communications and media subjects.
UCAS flags that “increasing UK 18 year old numbers from the 2021 cycle onwards” may mean that “unconditional offer-making may decrease in scale and impact in the longer term”. This looks like an argument that the rise in unconditional offers is at least partially attributable to increased competition for students.
Participation measured
It’s not clear why, but this time round we get information on access and participation by POLAR group earlier than information regarding the multiple equality measure (MEM), the latter of which is due in January 2020. There’s not much unexpected within the POLAR trends – each quintile has increased its entry rate by around one percent, and the ratio of quintile 5 to quintile 1 entrants has continued to fall in higher tariff providers particularly.
Scotland, of course, does these things differently. Participation in the higher and lower SIMD quintiles rises by less than half of one percent, but the three middle quintiles see a fall – with SIMD quintile 2 dropping by 1.2 percentage points. UCAS isn’t the whole story in Scotland as many young people apply outside of UCAS, in particular to HE in FE colleges – but apparently data sharing with the Scottish Government will be in place next year to present a more integrated picture.
2019 sees a record high – 12.6 per cent – of all accepted applicants declaring a disability. The increase in acceptances for prospective students with mental health conditions (19.2 per cent) is connected to a rise in diagnosis and a rise in awareness in wider society.
Internationalism and regionalism
EU acceptances have changed very little from last year (a 0.3 per cent drop) – meaning that we’ve not seen a huge change since the referendum. But non-EU international students have risen to the the highest number on record – 41,140. This 6.9 per cent rise over last year includes a 22 per cent rise in students from China and a 15 per cent rise in students from India.
Forty-four point five per cent of 18 year olds from London were accepted onto an undergraduate course in the 2019 cycle – with the capital region now well ahead of others (mostly clustered around a 30 to 35 per cent entry rate). The overall entry rate for 18 year olds in the UK is 34.1 per cent – though note as above the issue with data on Scotland.
Despite a 1.9 per cent drop in the number of UK 18 year olds this year, participation overall rose by 1.2 per cent – suggesting improvements to mature student recruitment.
Grade prediction accuracy maps on to A-level marking accuracy. Studies by OFQUAL show that marking is far more accurate for high grades (B, A, A*) than it is for low grades (C, D). Marking accuracy is also greatly influenced by where the grade boundaries are set.
Universities with tariff entry at the lower end of the grade profile could, if they wished to, combat the problem of inaccurate predictions by setting offers based on raw marks rather than grades.
The PQA proposal was an expensive, disruptive and over-complicated solution. As pointed out previously on WONKHE, there are no examples of countries that have both a genuine post-PQA (ie post-18) system of admission and better widening participation outcomes to those achieved by British universities. A tendency to compare Britain unfavourably with other countries was a feature of other aspects of the Corbyn agenda. It appears voters didn’t agree with this.