The introduction of the Small Business, Enterprise and Employment Act, as one of the final pieces of legislation of the last Parliament, paved the way for educational data to be linked to data from HMRC to examine the outcomes of leavers from education.
The Green Paper then stated that employment and earnings information derived from this linked data would be used as part of the common metrics that would feed into TEF for the purpose of informing applications for higher student tuition fees.
All of this has led to speculation that the venerable survey of graduate first destinations, which has been conducted by the sector for over 50 years and which is currently incarnated as DLHE, might be dropped. HESA’s announcement that outcomes data are under review has therefore excited a certain amount of comment. Why review something we no longer need? What is happening?
What the Small Business, Enterprise and Employment Act allows is the linkage of educational data to tax records, which means that, potentially, people allowed to access tax records can track someone’s educational journey through school, into HE, and then into the labour market and get accurate earnings information through their subsequent career. As the provisions state, this will be used to produce data on the returns to study of individual subjects and at individual institutions.
But there are good reasons that this isn’t the DLHE-killer some people have painted it as. One reason is data coverage. HMRC data will give us excellent information about salaries, and when linked to elements of student record data, this will potentially allow a vivid picture to be built of earnings by all kinds of characteristics – not merely institution and subject, but gender, ethnicity and disability status, amongst others. Previous educational experiences can be brought into the mix. But it’s not clear how good the information on other aspects of employment outcomes will actually be. Industry and occupation data are not guaranteed to be of the quality of the data collected through DLHE, and employment location may only be limited to head office data. Many very good objections have been raised about the use of salary data as a metric for university performance and salary data alone simply isn’t enough for the many stakeholders in university outcomes data. We need more and better information.
But there are more subtle reasons why it is advisable for the sector to continue to collect outcomes data. The first is the question of ownership. HMRC data is, obviously, not under the ownership of the HE sector. Indeed, such are the restrictions on how the data might be used it is possible – even likely – that nobody in higher education will ever be able to access the full raw data. We may be able to obtain extracts, partial datasets or processed data – the details are still to be worked out – but anyone dreaming of using the full data will probably have to moderate their ambition. On one level, this is not an issue – indeed, it may be of value for HE to be assessed using data that nobody can accuse institutions of manipulating in any way. But data that is not administered or controlled by the sector cannot be modified or adapted to suit changing needs. Our own outcomes survey allows us to ask questions at a national level that can then be used by the sector as a whole or by individual institutions that may not be directly related to earnings.
The second issue is more subtle. DLHE data is collected with the informed consent of the participants. The same is not true of linked HMRC data, collected, as it is, by Act of Parliament. We can’t very well ask for students to actively consent to their tax data being used by institutions – I am no expert, but I can’t help feeling it would be a pretty tough sell – and so data that comes from consenting participants may still be valuable.
DLHE has certainly come in for a good deal of scrutiny of late. It was reviewed relatively recently, in 2010, but the subsequent rapid expansion of public metrics has led to the data no longer meeting the needs of a much-expanded group of significant stakeholders. The 2010 Review largely drew upon the needs and experiences of the careers and employability community who were (and in many cases still are) responsible for collection and were the largest consumers of the output of the survey. Careers professionals remain an important stakeholder group and one whose experience of collecting and using outcomes data should not be sidelined, but this information is also crucial to other sector stakeholders and any data must meet more diverse needs.
And this is our opportunity. A space remains for a sector-owned, population level outcomes survey, collected from an audience with their knowledge and consent, which can address the current sector needs. The most troublesome aspect of DLHE – the collection of salary data – is no longer vital. Everything else, from the reference date (we will probably move away from six months) to the sample coverage, is up for grabs. Shall we keep a focus on employment outcomes? Should we take a more qualitative view? Can we use a survey of this nature to examine more fundamental questions about the individual’s perception of the value of higher education?
2016 will almost certainly see the last DLHE as we know it, but the long story of UK higher education outcomes data is not yet over.
Excellent article. Two thoughts: • We should also consider the international angle: what graduate career outcome metrics might allow international comparisons. • I’m never comfortable with the idea of salary being equating with career success. (If I learnt more, perhaps I might be.) It’s rare that people base career choices on salary alone and often, they do so in spite of it. Think of social workers, nurses, charity workers, even academics for that matter. While the potential to use HMRC data will be a huge boon to analysis of the labour market and the effectiveness of education in meeting that market’s needs,… Read more »
Very good point about the international angle Johnny. I didn’t cover that as I think we can all agree that DLHE’s international coverage is not that great, and because although it’s a consideration in the new review, we’re pragmatic enough to realise that collecting really good international data would be very demanding of resources. Salary is not a good measure of career success. The ways individuals regard salary in career decision making is complex and not reducible down to ‘higher=better’. I agree that the HMRC data will have real value – although I think the long-term value is more likely… Read more »
I agree with you Charlie, the HMRC dataset is no replacement for DLHE. There are four groups in the population: 1. those that pay tax/claim benefits and are graduates and can be matched to the HESA student return, 2. those that pay tax/claim benefits and are graduates and cannot be matched to the HESA student return (overseas graduates as an example) 3. those that pay tax/claim benefits that are not graduates and cannot be matched to the HESA student return 4. those that do not pay tax/claim benefits. Since many graduates cannot be linked to the HESA student return (group… Read more »
The weaknesses of DLHE however are legion: 1) It’s a snapshot only a notional six months after graduation, many graduates have yet to get into their preferred area of employment this early in their post-graduation career; Longitudinal surveys at least several years later (and repeated) are necessary to get an accurate picture of the changing nature of graduate employment. 2) self employment and ‘portfolio’ jobs (multi-employers) are much more widespread than in the past especially for some sectors, such as creative arts, and salary data for the self-employed only really makes sense if it’s compiled on a 12 month tax… Read more »
Mike To take the issues in order. 1+2. The point is taken to an extent, but rather than explaining the issues with six months, we need to hear specific, robust arguments about why another reference date is obviously better. These need to take into account that the further you move from graduation, the more of the sample will become difficult to reach, the more that data will cost to collect and the less the coverage. As LDLHE shows, a survey after 3.5 years can be less useful than one after six months if there are not enough people in it.… Read more »
Thanks for you detailed comments. I take the point about the 1990s cohort studies. Perhaps I should have made clear that I think the situation has changed with the passing of the SBEE Act and that in order to be any use any new pilot would have to work with HMRC to see if we can join up tax and institutional data in an upwardly scalable and cost-effective way (clearly LDLHE is too small anyway, and HMRC tracking, especially given the future importance and long term implications of student loan repayments, in the future should make it easier to track… Read more »
Paul, I agree with you on much of that, and you know that I feel strongly that as a sector we have not always appreciated DLHE enough or used it as effectively as we ought. There is a lot of value in the dataset and a lot of uses it can be put to within and outside the sector. The place where you take issue with me – I offer two observations. The first is that I was on the last DLHE Review, and the review was composed almost solely of careers professionals. I think that had we had been… Read more »
Hi Charlie
Fun times! I was at the HESA’s first strategic agenda setting meeting for this and the international issue was brought up a few times. Possibly universities are going to be left to their own devices when it comes to measuring international student success (not that salaries are remotely comparable in the first place). And I’d go further than salary not being a good measure of careers success – the idea of ‘grad/non-grad’, a tail which has been wagging the statistical careers dog for for too long, is an equally dodgy metric…
Hi Michael
I think the feeling is that the semi-voluntary basis on which current international DLHE data is collected has potential if we can get more uniformity of response rates. It can take a lot of resource to collect international data and unless we suddenly acquire a lot of money from somewhere to do the work (you don’t know of any rich benefactors with an inordinate interest in international student outcomes?), we’ll have to carry on with the current level of resource.
I may have more to say on the grad/non-grad job question in the reasonably near future….