Life is full of surprises.
Sometimes the things we thought we knew as simple unassailable truths turn out to be wrong or, perhaps more often, complex, in ways we couldn’t imagine.
We all know that students go to university and study on courses. The whole narrative of post-compulsory education is based on the idea of a student doing a course, and government policy is increasingly obsessed with them: good courses, bad courses and helping students choose the right course. But what is a course?
A bit of history
In 2011 I celebrated two decades working in HE data. At that time the big news in sector data was the Key Information Set (later rebranded UniStats) which sought to improve the provision of course-level information to applicants. It turned out to be a huge challenge since the policy behind KIS was based on that simple unassailable truth of students studying on courses.
Knitting together course-level data around entry criteria (UCAS), student satisfaction (NSS) and graduate destinations (DLHE) was a massively complex issue because there was no common understanding of what a course is. This had never been a big problem prior to the KIS but with the increasing emphasis on course-level information the question needed to be unpacked and better understood.
A small project ensued, looking at what a course is at different institutions and how the concept changes at different stages of the student lifecycle. A brief report – aimed at policy-makers rather than data geeks – was drafted and published in December 2011.
Why the question is hard to answer
“Course” is defined by providers. We concluded that:
there is a broadly common model of a student registering on a thing that is made up of components (which might include sub-components) and which is defined by some sort of outcome
So the definition of a course within an institution will depend largely on the academic regulations and structures of that institution and is often influenced by regulatory bodies linked to specific disciplines and professions. It often varies significantly within an institution as different disciplines take different approaches to pedagogy and to the type and amount of flexibility that students have. In some cases it is normal for the qualification a student is aiming for not to be established until the student has completed two (or three in Scotland) years of study.
Across the student lifecycle the differences can be even more stark. An institution might have a large number of courses in the UCAS and UniStats data that define specific entry routes to what is actually a single course structure within the institution. This could be reflected differently again in the SLC course data (where data is optimised for the administration of tuition fee payments) and the HESA data (where the definition of a course depends on a complex and esoteric set of factors, largely in the hands of individual institutions).
The data professionals who link and analyse these datasets understand this – so work that links back to the HESA student record (like NSS results, destinations data and LEO) uses broad subject groups rather than having any pretence to analysis by actual courses. But a course can have many (previously 3, now 5) different subject codes – so data for a course can be split across many subject groups – and a subject group can contain data from many different courses delivered in various parts of the provider.
This many-to-many relationship is why the Subject TEF could never say anything useful to an applicant about a specific course. And while we might breathe a collective sigh of relief that the Subject TEF appears to be lifeless – possibly dead – the worrying thing for me is that it ever got as far as it did.
Up to date
Now, nearly a decade after we unpacked “What is a Course”, the paradigm of students on courses continues to dominate policy, as attention turns to the LEO data and the dogmatic attack on low value courses, as demonstrated by this babbling nonsense from one of our recent Secretaries of State.
Much has been written about why graduate salaries say nothing about teaching quality. But beneath this there is an awkward truth about the debate around low quality courses; given the diversity of the sector and the complexity of the student journey it is impossible to build a data model that provides true course-level information.
Life is full of surprises, isn’t it?
Best endeavours of superb teachers cannot plug the gaps in the knowledge and training of students with very low A-level results. Low entry requirements (irrespective of discipline) is, in my opinion, a sufficient condition for a course ( regardless of how we choose to define it) to be defined as ‘low quality’. There will be some who will deny this truism. They would have to explain however, assuming for simplicity uniformity in the quality of teaching, by which process two cohorts with, say, 2 D’s and 3 A’s average entry will attain, by the end of the third year of… Read more »
There’s substantial evidence that courses with lower entry requirements often produce better salary outcomes than those with higher entry requirements. Explain – as you say – that?
A-level performance leads to better salary outcomes: https://londoneconomics.co.uk/wp-content/uploads/2015/03/London-Economics-Report-Returns-to-GCE-A-Levels-Final-12-02-2015.pdf A-level attainment leads to better university outcomes: https://dera.ioe.ac.uk/id/eprint/19811/1/HEFCE2014_03.pdf Highest earners come from the higher A-level performing cohorts: https://wonkhe.com/blogs/the-terrifying-return-of-the-uks-worst-higher-education-courses/ (Hate to use your own article against you, table 3, this is a weak trend but it cannot be ignored) In your own articles on the topic, the comparison of courses can seem like there is no difference between courses and universities. However, your general trend of direct comparison is one that takes a low salary outcome course from a “high-status” university and a high salary outcome course from a “low-status” university and use… Read more »
I enjoyed this article greatly. My favourite bit was trying to guess which bit of “babbling nonsense from one of our recent Secretaries of State” you were linked to without clicking the link, because there’s just so f***ing much. Subject TEF is a car crash. It was always going to be more misleading (for students) than actual TEF, as it pretend to tell stories about courses while never having a hope of doing so – we’d have swapped an institutional rating (which, even if accurate, would not be accurate for every course, but was at least *called* an institutional award)… Read more »
I’d very much like to see the evidence. Thank you. Perhaps that: https://www.thecompleteuniversityguide.co.uk/league-tables/rankings
Try here to start with https://wonkhe.com/blogs/britains-worst-higher-education-courses/
Good piece, but you’re still a newcomer Andy and it’s actually got a lot simpler in the last 20 years than it used to be back in the Thatcher days in English FE/HE, thanks to the virtual wiping out of non-vocational and free standing part time professional “courses”. I was working in finance originally and remember turning up to my first day release class in Economics at the Oxford College of FE, thinking I was preparing for the chartered secretaries qualification. It turned out the majority of students in the class were taking the exams of the “Chartered Institute of… Read more »
I like this very much 🙂