Is TEF making a difference to early indications of applicant interest?

Prospective students seek information about universities and colleges in a number of ways – from the institutions themselves, from the government’s Unistats platform, and increasingly from league tables and other commercial information sources. Hotcourses Group is behind three major sites: the Complete University Guide, Whatuni and Postgraduate Search.

Hotcourses Group has given us sight of their live data, and by analysing the ways in which institutional pages are is accessed – looking at the year between September 2016 and September 2017 (which amounted to more than 37 million institutional page impressions) – we can begin to understand how prospective students are selecting the right institution for them.

To everything, there is a season

As you would expect, there is a cyclical pattern underlying the use of these sites. August and September (clearing time, and the opening of the UCAS mainstream applications) are high points, with December being the quietest month. January sees a smaller peak as mainstream UCAS applicants rush to meet the mid-January deadline for applications.

Perhaps surprisingly, there is no uptick for accessing information about individual institutions for April, when the Complete University Guide for each year is published, although the site visits for the day in question rise as you might expect. Indeed, interest appears to fall from the previous month. Although newspapers and vice chancellors see the launch of a league table as a huge event and views of the rankings bear that out, it appears that those who do obsessively follow the rankings scene are not the same as those who are interested in detailed information about universities and colleges.

League of interest

One interesting upshot of this data is the ability to sort institutions by the proportional interest each receives in a given month. For September 2017, the proportionally most “interesting” universities were:

I’d excuse you for not being enthralled by that particular list – these are institutions we already know to be popular, and this data merely proves they are also popular on Hotcourses Group sites (though there are a few surprises – where is Oxford, for instance?) For reference Cambridge pages were served to 2.488% of the total number of users in September 2017, which works out at 25,689 users who saw at least one page.

If you look at a sum of users over the twelve months between September 2016 and September 2017, the picture changes only slightly. Even looking at a view across the year you do not see interest, for instance, in institutions known to be popular in Clearing. The below table shows the relative position of universities and their interest, versus their ranking on the Complete University Guide (CUG).

Growing your share

Where it gets much more interesting is looking at change over time. For instance, while we would expect Loughborough to be popular, it has become less popular over the course of the year – despite receiving a TEF Gold, having stellar research performance and a huge number of high profile sporting events.

TEF, more generally, was designed to shake up the rankings space, and drive interest in institutions based on a measure of “teaching performance” in contrast to a league table focus on research, and the more nebulous ideas of ‘prestige’ and ‘academic esteem’. You would expect, if this were correct, to see a rise in interest for TEF Gold institutions, and a corresponding drop for those holding TEF Bronze.

Of course, this is only one factor that will affect the level of interest in an institution. A well-judged marketing campaign, good (or bad) press coverage, or simply a high-quality and attractive range of courses and facilities can have an impact on recruitment, and thus we would expect a similar impact on interest measured in this way.

With the top six growing interest all having a Gold, there might appear to be (limited) evidence that TEF awards may be having the desired effect in the UK. Looking over all institutions for which data exists, the pattern is starker.

There’s no correlation at all for Gold award holders – institutions in this category are very as nearly as likely to have seen a decrease in interest levels as an increase. Recall, of course, that of the top ten institutions with the highest overall proportion of interest none held a Bronze TEF award. And interestingly the both Solihull College (the FEC with the highest growth in interest) and the University of Law (the AP with the highest growth in interest) hold Gold TEF awards.

The Bronze category is perhaps a greater concern. We see 69% of Bronze award holders experiencing a drop in interest over the year. It should be remembered that the DfE line is that Bronze institutions are still high-quality (but not highest-quality) institutions that meet stringent UK quality standards. But there are reasons to suspect that there is no causation – very few institutions in this category shows a drop in month-on-month interest after the TEF results were released in June.

The changes between September 2016 and September 2017 reflect the run-up to the first mainstream UCAS cycle after the first TEF awards – if you were going to see changes in prospective student behaviour you would expect to see it in this comparison, as keen students begin to research potential courses and institutions. The fact that you don’t suggest that TEF may not have had the hoped-for impact on the home market for students, at least so far.

But what does matter to prospective students? There are various Hotcourses Group sessions at Wonkfest17 that will attempt to answer this hugely important question. And, for my part, I’m looking to getting stuck into more of this data. Institutions like Coventry, Huddersfield and South Bank are clearly doing something right – but what?

This article draws on data from 278 institutions for which Hotcourses Group held data. Raw traffic figures represent users who saw a page specific to that institution. Therefore, one user who saw multiple pages would be counted as multiple users.