Are there providers where graduates tend not to repay their loans?

The Times is especially excited about a release of data (following two Freedom of Information Act requests from shadow education minister Neil O’Brien) by the Student Loans Company, purporting to show which universities produce graduates most likely (and least likely) to repay their student loan debt.

As a brief reminder, you usually begin to repay your student loans (of any sort) after what is known as the Statutory Repayment Due Date – usually the start of the tax year after you complete your studies. At this point, provided you have an outstanding loan balance and are earning more than the repayment threshold (that’s £25,000 for the current undergraduate loan system – it is different for others) you repay 9 per cent of the amount you earn above that threshold.

The data released purports to allow you to see the average balance (the amount of loan left to pay) and the average amount that has been repaid. Dividing the latter by the former has allowed The Times (or someone) to calculate the average amount of loans that have been paid back. According to O’Brien “The variations between institutions are absolutely enormous”. And according to HEPI’s Nick Hillman “Questions should be asked by regulators when so few students at some institutions seem to be paying their way”.

Can you safely make such determinations from the data as presented? Well no, of course you can’t. Here’s a couple of caveats that appear to have been missed in the press coverage:

• The data includes all student loans (that’s UG fee loans, UG maintenance loans, PG loans, and everything else) going back to 1998 (the year student loans were invented)
• The data includes information from four discrete funding systems (England, Northern Ireland, Wales, Scotland) that have diverged markedly over time.
• A student is assigned to the last provider they studied at – loans amounts are not disaggregated by provider. So if you did UG at the University of Kent, a Master’s at Birkbeck, and a PhD at Plymouth your debt and repayments are shown related to Plymouth.

I’m sure you don’t need my help in spotting many issues here, but as a starting point:

• There’s no attempt to control for the amount borrowed per student. Students from less well off backgrounds will borrow more maintenance loans, students on longer courses will borrow more to pay fees and maintenance than those on shorter courses.
• There’s no attempt to control for subject of study, level of study, subsequent employment, sex, region of residence or even completion (the data will include students who did not complete their course). We already know from the LEO data releases and accompanying IFS research that all of these things are factors in student earnings.
• Even the by university framing doesn’t hold up – there are duplicates in the data, and Oxbridge colleges are included separately because “This will depend on how an HEP has added their details/courses to the Courses Management Service”
• There’s no attempt to control for the number of cohorts. Clearly students who graduated in the early 00s will have had more time to repay loans than those who graduated last year – if a provider only became eligible for loan-backed fees or student maintenance recently clearly students will have more money still to repay.
• Though there is some attempt to control for sample size (if there are less than five students who have their loan balance assigned to an institution it is redacted) we are still faced with the impact of the choices of small numbers of students being disproportionate at smaller providers – again, this is just how averages work.

When Neil O’Brien tried doing this via a parliamentary Written Question the redoubtable Janet Daby had a fair stab at explaining some of this when she released England only data of a comparable level of utility. It’s a shame the shadow minister didn’t pay attention.

When I saw the story I was looking forward to plotting some new data. Having looked at the data, I have chosen not to plot it – I don’t want to draw further eyes to such useless information.