Making an impact?

What do social policy at UCL, civil engineering at Cardiff and communications at LSE, have in common with psychology at the University of East London, history at Hertfordshire and English at Bedfordshire? Answer: they are all making a big difference to our lives, according to the 2014 REF.

These are among the forty university departments judged to be 4* for impact across 100% of their submissions. This factoid hints at a long-standing concerns about the impact factor – can we really say the difference being made here is comparable across such very different fields?

Perhaps more pertinently, let’s consider what is being claimed here – that everything these forty departments does achieves (and we quote): ‘outstanding impacts in terms of their reach and significance’. No small feat, if we really believe it.

The reality is that what is being tested is not everything they do, but rather everything they submit. The other thing that unites the six departments above is that they all put fewer than 15 FTEs into the REF. This may tell us more about the benefits that can be garnered by making judicious (and no doubt internally difficult) choices at submission point. It also makes achievements like that of the medicine and public health departments at Bristol (both rated 100% 4* with 84 FTEs and 75 FTEs submitted respectively) look genuinely stunning.

Spare a thought for the five departments (we won’t name them, but you can look) that were rated 80% or more unclassified for impact (i.e. ‘the impact is of little or no reach and significance; or the impact was not eligible; or the impact was not underpinned by excellent research’) and all put fewer than ten FTEs in for submission. What on earth were they thinking?