LIVE: Longitudinal Education Outcomes data released

Thanks for following our live blog today. We haven’t even begun to scratch the service of this data release, but we hope we’ve been of service to make some high level sense of it all.

There will be plenty more analysis and comment to follow on Wonkhe in the following days, so watch this space.

We’ve got two new blogs up for you to chew over this evening:

• There are plenty of interesting insights and perspectives to be gained from a day’s worth of analysis of the the new Longitudinal Education Outcomes dataset. David Morris suggests some winners and losers from this release.
• Universities need to get to grips with the politics of Longitudinal Education Outcomes data, and fast, or suffer the consequences, suggests Ant Bagshaw.

Psychology

Psychology finds itself at the lower end of the subject earnings tables, and there are a large number of institutions here whose courses appear to produce only a minimal or negative premium.

Social Sciences

Subjects Allied to Medicine

This is a difficult one for comparisons, with a huge range of medical careers and professions fitting into this category.

It’s clear that someone, somewhere, is going to be misguided enough to produce a LEO institutional league table – ignoring all of the caveats that we have been carefully adding to the live blog throughout the day.

Without taking into account coding discrepancies, region, ethnicity, gender, subject balance and many other well documented issues, such a table would be almost meaningless. It would be used to rank institutions unfairly and without the nuanced understanding that a representation of each institution and their unique role within the graduate economy would be.

We have, however, modelled one internally using the mixed gender 5 year data set. We’re not going to release it in full, because we feel it would be irresponsible to do so. But as a taste, the top 10 includes:

1. The London School of Economics and Political Science
2. Imperial College of Science, Technology and Medicine
3. University of Oxford
4. St. George’s Hospital Medical School
5. University of Cambridge
6. University College London
7. King’s College London
8. The University of Warwick
9. University of Bristol
10. The University of Bath

What connects these institutions with a high average graduate salary after 5 years?

• For a start, eight of the top ten have a medical school. We’ve seen in our more detailed analysis that medical students tend to have high earnings – these are reflected here in institutional figures.
• Eight out of 10 are Russell group members – a history of and reputation for selective admissions is clearly a factor, as we already know prior attainment is in many cases a good indicator of future earnings.
• And five of the top 10 are London institutions – London salaries of course being higher (and often weighted higher) than others in the UK.

None of these factors necessarily make these institutions better than any other – there are plenty of great universities that do not have a medical school, or are not based in London, or do not have a recruitment policy so focused on prior attainment.  And, of course, past performance is no guarantee of future earnings.

Wonkhe’s Mark Leach and Catherine Boyd discuss the gender pay gap among graduates that’s becoming apparent in the LEO data.

Mass Communication and Documentation

No doubt some parts of the tabloid media would love to seize on ‘media studies’ as evidence of ‘worthless’ degrees, but as this data shows, earnings outcomes don’t differ a great deal from humanities subjects such as English and History. Many top-tariff institutions don’t offer this subject, so Loughborough tops the pile.

Mathematical Sciences

Medicine and Dentistry

The most boring graph of the lot, and thus, in some ways, the most interesting. Doctors and dentists salaries don’t tend to vary a great deal, hence the fairly stable line here with a very small range of outcomes. This is one of the few higher education ‘league tables’ where you’ll find Oxford and Cambridge towards the bottom…

Nursing

Another one with very small variance, the University of Portsmouth finds itself producing the highest earning nurses.

Physical Sciences

As you would expect, average earnings for most subject groups are higher within the Russell Group. In 18 subject groups out of the 23 presented, the highest average graduate earning after 5 years (where men and women are considered jointly) can be found in Russell Group institutions rather than in other mission groups.

Exceptions include Million Plus for Architecture, Education and Nursing, the University Alliance for Medicine (a strong showing by the University of Brighton, in partnership with the University of Sussex) and non-aligned institutions for Veterinary Science (likewise a win for the Royal Veterinary College).

In a press release the Russell Group said “From psychology to social sciences and law and from computer sciences to engineering and physical science, Russell Group universities outperform in the graduate earnings table”. In fact, the news for them is even better than that. But of course, as we have stressed, stressed, and stressed again, this as much to do with Russell Group universities’ student intake as it is with their performance. 

One issue that keeps emerging with these figures is the variability in which institutions apply JACS codes to courses. Although the shortly-to-be-superseded JACS system allows for a great deal of specificity (at four hierarchical levels), institutions may assign their courses to different codes depending on their own requirements – which could include anything from institutional data use needs to perceived advantages for funding or reputation.

The HEDIIP project, which resulted in the development of the new HECoS (and HECoS CAH) classifications, conducted a review of the way JACS was used. One memorable reported finding was:

“A respondent to the [HEDIIP] review survey cited one particularly anomalous example from the Unistats website where their BA Fashion Marketing course returns satisfaction statistics from Materials & Minerals Technology and work and/or further study statistics from Engineering & Technology. Without going into the detail of whether this is down to the institution’s own coding or the KIS algorithms, it is clear that such data is of little use (and indeed misleading) to potential students and that the current classification system is not lending itself to enabling clarity.“

When we see unexpected data within the LEO stats, it usually boils down to an unexpected coding decision by an institution.  As we move to a more data-driven slant for institutional and public policy, these coding decisions will see increasing scrutiny in the months to come.

The use of prior attainments stats in LEO hasn’t changed much since December. In both LEO releases they use NPD (National Pupil Database) to access top three A Level results to place them into four groups (upper quartile, middle 50%, bottom quartile, no data) what are four groups). There have been calls in the survey to include other Level 3 qualifications, such as BTEC. However, the problem this time round is LEO includes Scottish and Welsh universities. As LEO is reliant on NPD data, which is English only meaning they can only calculate A level results studied at English Schools. Consequently, there are no (or very little) prior attainment stats for Scotland or Wales.

Another helpful interactive chart from Jonathan Boys.

More subject comparisons. Remember to click the graphs to see them larger.

Engineering and Technology

The oil industry is no doubt the main reason for the University of Aberdeen’s place at the top of this table, though remember that this data is from the tax year 2014-15, just before the oil industry went into a slowdown. If future years show Aberdeen fall down this chart then it will only further underline how dependent LEO results are upon wider economic movements.

English Studies

Interestingly, English was separated out from other Languages after the test release back in December. The reasons why become clear when you compare this chart to the Languages chart below – English graduates tend to earn significantly less than those who take Languages. There is a worrying tail here of courses where graduates are struggling to earn more than the median wage for 25-29 year olds, though gender may be a factor here, with women making up the majority of English students.

Historical and Philosophical Studies

Staying in the humanities, we see a similar trend to English, with fairly solid earnings in the top few institutions but a fairly long tail of courses struggling to show a premium at the far end (even more so that English).

Languages (exc. English)

Things looks healthier for Languages, where clearly there is a greater amount of labour market demand than there is for other humanities courses. Perhaps LEO may be the public information required to arrest the well-documented decline in young people studying modern languages in schools and universities?

Law

Law was the one subject released in this format in the December test release, and this graph continues to show a high-end ‘peak’ in law graduate salaries (presumably those entering the profession), and a longer tail (presumably of those that do not).

We break for this quick announcement from Wonkhe’s Editor and Director Mark Leach:

I am delighted to announce that from this morning, David Kernohan will be joining Team Wonkhe as Associate Editor. David has supported and written for Wonkhe since our inception, both as a featured author and supporting analysis behind the scenes. Many in the sector will already be aware of him as one of the sector’s most distinctive voices. David is already getting stuck in to LEO analysis and has big plans for serious wonkery over the coming months.

Read more on the Wonkhe Team Blog.

Sorana Vieru, NUS Vice President for Higher Education, has pointed out other determinants of graduate earnings

Parents’ income, race and gender are also great determinants of graduate outcome, not just where and what you study https://t.co/KoEnxurOAl

— Sorana Vieru (@SoranaBanana) June 13, 2017

UUK has launched a critique of the emphasis on graduate earnings

Success in higher education shouldn’t simply be measured by how much graduates go on to earn: https://t.co/Gf4X8rZTuo pic.twitter.com/H2PJXFKCV6

— Universities UK (@UniversitiesUK) June 13, 2017

Picking up on the gender pay gap is GuildHE’s Cat Turhan

Anybody still convinced that the Gender Pay Gap is a myth should look at this year’s #LEO data – it’s genuinely shocking!

— Cat Turhan (@cattyfantastic) June 13, 2017

And Kate Wicklow highlight the lack of data on multiple groups of individuals

No self-employed; or students with jobs; or those on mat/pat leave included in the #LEO earnings data. Its not exactly representative…

— Kate Wicklow (@KateWicklow) June 13, 2017

The main analysis for today’s release from DfE has some interesting, and depressing, statistics on the distribution of median earnings between men and women in each of the subjects five years after graduating (pg. 11). It seems for women, English Studies is the only subject that offers earnings that exceed men (but only just). The document tells us “for all subjects except English Studies, male median earnings exceed female median earnings at more than 50% of institutions offering that subject. In 12 subjects, male earnings are greater than female earnings at more than 75% of institutions.”

However, at the last experimental release of LEO back in December 2016 we discovered that the gender pay gap starts immediately after graduation. We’ll be looking at how the median earnings between median women fair one year after graduating throughout the day.

————————————————————————–

Unlike the December release, there is no inclusion of ethnicity data in this version of the LEO. However, in the main analysis from DfE they discuss data matching and match rates for graduate characteristics and outline issues with Chinese forenames and surnames switching. Therefore, we know they have the data but where is it?

The most likely answer is that because of small numbers at subject level we have statistically insignificant data on ethnicity. LEO suppresses information for cohorts of fewer than 11 graduates. Unfortunately, at institutional level we would have this data but it hasn’t been released in this version. Possibly because it would results in some misleading headlines. Regardless, we are working hard at Wonkhe HQ to pull this together.

We continue to compare different subject outcomes, slowly making our way through the alphabet.

Click on each graph for a larger view.

Computer Science

Not many surprises here, and Computer Science tends to be at the higher end of the ‘general’ subject salary outcomes. Things get a little steeper towards the higher end of the sector, but all in all this is a subject area without a bottom end ‘drop off’ in salary outcomes.

Creative Arts and Design

This was always going to be a tricky one, with a big caveat that many Creative Arts graduates (in some institutions up to a fifth) are self-employed and may not be captured by the data. Nonetheless, the earnings outcomes for some of these courses are a bit bleak. A we also have a surprise chart topper: the Bournemouth University pips Oxford at the top end, perhaps a quirk of subject coding?

Economics

Some big variances here, suggesting a diverse array of graduate destinations for Economics graduates both between and within institutions. A select number of LSE economics graduates appear to be doing well…

Education

Some curious results here, with a fairly flat middle range of earnings outcomes given that most graduates will be working for the same employer. The death of collective bargaining in the teaching profession in England may begin a gradual widening of these outcomes over time.

More to come…

Mark Leach and Ant Bagshaw talk about the big challenges that LEO and other new datasets pose to universities and a first take on this morning’s results.

There hasn’t been much in the way of official reaction from institutions or the wider sector. However, the debate on social media has been heating up.

Paul Greatrix points out the radio silence.

Slightly disappointed at lack of big statements about #LEO (from individual institutions) & that hashtag is mixing in lots of astrology too

— Paul Greatrix (@registrarism) June 13, 2017

London Economics has been quick to point out that graduate earnings are likely to be as a result of the individual, not the qualification.

LEO analysis looks like wage returns to individual in possession of qualification – not the qualification per se. https://t.co/7oNgyrI11b

— LE Education (@LE_Education) June 13, 2017

Million Plus’ Pam Tatlow points out some of the caveats.

5 year LEO data based on 2008/09 graduate cohort & does not properly include self-employment or mature students – needs govt health warning

— Pam Tatlow (@millionplusCEO) June 13, 2017

Newsnight’s Chris Cook is being cheeky…

Unclear if historians are allowed to declare pub quiz winnings in this table. pic.twitter.com/ZveAOfX7jB

— Chris Cook (@xtophercook) June 13, 2017

Jonathan Boys – a Senior Researcher at Careers and Enterprise has made a nice searchable Tableau from this morning’s data.

We’re gradually crunching the graphs for some of the most interesting aspects of today’s release. Bit by bit, we can create graphs comparing the subject outcomes for different institutions. It’s particularly interesting to note the different variations between institutions for different subjects, as well as the variations in outcomes within particular institutions’ cohorts of graduates.

Click on each graph for a larger view.

Agriculture and related subjects

Only a relatively small number of universities provide course in agriculture and related subjects. As we will see, compared to other subjects, the salary variances are relatively small.

Architecture, Building and Planning

Anglia Ruskin clearly has a high value course in this subject area, which throws up some surprises, such as powerhouse Cambridge coming midtable. One suggestion in Team Wonkhe for this is a lack of real comparability between Architecture and Town Planning courses, which tend to lead to very different careers with different earnings trajectories, particularly for young people.

Biological Sciences (exc. Psychology)

There’s a lot of institutions on this list. One particularly interesting pattern, which as we will see is similar to other subjects, is the ‘far end jump’ for Oxford and Cambridge (and in this case, UCL), which might tell us a lot about ‘elite’ salary outcomes and their educational backgrounds. Those extremes aside, the gradient of median salaries is actually not that step, and most courses here appear to deliver fairly solid salary outcomes.

Business and Administration Studies

Another subject with an ‘elite spike’, the very highest earning business graduates from Oxford go on to make silly money only five years out from graduation. 

One of the key findings both from the earlier experimental release of LEO data and the linked IFS analysis was just how good a predictor prior educational attainment (pre-undergraduate) was of future earnings. Now we have the full data this gets even more interesting, as the relationship is far stronger after five years in some subject areas than in others. This pattern is of course more meaningful in subject areas with larger cohorts.

The entry requirements predictor is strongest in:

• *Business
• Biosciences (excluding psychology)
• *Computer sciences
• *Economics
• English
• History
• Languages
• *Law
• *Physical Sciences
• Psychology

(see figure 6 onwards of the report)

So what other factors could be at play? For half of the list, the answer could revolve around gender. All of the subjects starred above had a majority of male students in 2006-7 (for 2008-9 graduation). We already know from the experimental release that gender pay disparity starts immediately after graduation, and that a small number of (usually selective) institutions tend to dominate very high pay. So we can assume that subjects areas dominated by men could see more very high pay for male graduates of selective institutions.

But this is, of course, only half the story.  We see sciences, humanities and languages on the list where the gender mix is more along the lines of HE as a whole. None of these subjects are linked to clearly defined professions with standardised progression routes, none of these subjects (with the possible exception of computer sciences) correlate strongly to an expectation of self-employment for “successful” graduates.

As part of its suite of documents released today with the LEO data, the Department for Education has published a response to its survey on LEO following last December’s data release. The consultation received 24 responses from providers, representative bodies and other interested groups.

The most pressing concern for the sector is the contextualisation of the LEO data. DfE’s survey response shows that there were calls for concurrent data released to show salary data relative to regional variations and also benchmarked for similar student cohorts (as TEF is):

21. The response to the questions prompted a large number of suggestions. Generally, respondents felt that including information from the Destination of Leavers from Higher Education (DLHE) and the Teaching Excellence Framework (TEF) was important. There was also a number of suggestions for including regional information on salary differences, the average graduate salary along with regional labour market characteristics and sector benchmarking as a means of contextualising the employment and wage data.

While we know that the social and financial capital of students prior to university had a big impact on their employment prospects, there is the potential – perhaps inevitably – for LEO to impact on student recruitment strategies. With variations by gender and ethnicity, the danger of misuse of the LEO data should be a significant concern:

36. One respondent argued that the large variations in outcomes and earnings by subject and characteristics should be measured against benchmarks rather than raw measures. They feared the data would offer a perverse incentive to avoid recruiting groups with characteristics that are associated with lower employment Rates.

The response promises, helpfully, that there will be more information to follow:” DfE has also commissioned research which will use LEO data to look at graduate outcomes after controlling for the different characteristics of graduates.” This should be welcomed, but in spite of the survey responses, and the widely-discussed problem of presenting the data in precisely the way today’s release does, the heat of the debate may still be around particular institutions and particular subject areas without sufficient nuance.

A table showing earnings as grouped by subject has already been doing the rounds on twitter. As it happens, this data was already made available in the December release.

Last December’s release also showed how much some subject outcomes can vary dependent on graduates’ attainment prior to university.

The meat of today’s LEO data release will be the analysis by institution; that’s the data of greatest interest to the sector given the earlier publication of data on earnings by subject.

However, the Department for Education’s explainer on the release goes into significant detail on the subject areas. As those of you familiar with subject coding know, the 23 subject titles in use cover a wide variety of programme types and subjects. This therefore masks variation by factors such as whether any given course of study has a vocational focus.

Woven throughout the LEO analysis is the caveats of what’s in, and what’s not, a theme that we’ll certainly come back to. The data release doesn’t include alternative providers, but the University of Buckingham is in. The assessment of students’ prior attainment covers their A-level scores, but doesn’t show other qualifications (e.g. BTECs). And prior attainment is also only available for English HEIs.

The biggest caveat of all, glibly expressed in the DfE document is that while we have the data by institution and subject studied: “There are a number of factors that can influence the employment and earnings outcomes of graduates beyond the subject and institution attended.” And there’s a similarly important note about regional variations:

“There are also well-documented regional difference in pay across the UK. We have published the region that each university is located in. However, we do not have the current address of the graduates, so we do not know whether they have stayed within the region where they went to university or have moved to a different region to access a job with higher pay.”

The debate will rage about whether, given these known issues, it was ever the right thing to do to release the data in this way.

Almost bang on time (a rarity for government releases in our experience), the data has been released.

Tuned in but still not sure what LEO is exactly? Have no fear! We have the ultimate beginners guide for you to digest.

Furthermore, the Department for Education were kind enough to deliver an experimental release of a portion of the data last December. We covered some findings from that release: “Prior attainment, gender gaps and other lessons from LEO”.

We’re expecting the DfE release of today’s data imminently. More to come shortly.

Good morning. Today is a big day of higher education wonks up and down the country. For the first time ever, employment and salary outcomes of graduates, broken down by their university and their course, will be made publicly available by the Department for Education and the devolved administrations. The release ties together data from the Student Loan Book, HMRC and the Department for Work and Pensions.

The release will give us an unprecedented insight into the destinations of different graduates, and could pose some challenging policy and reputational questions for parts of the sector. There will be plenty to chew from, and we’ll be bringing you our reflections, insights, analysis and comment here all day.