The keenly awaited second set of REF decisions were published in late November 2017 after further consultation focused on staff submission and research outputs. Some previously identified and sensible decisions were reported, but not in every case. Although the submission of all staff with significant responsibility for research was indicated, it is confirmed that institutions which do not have all academic staff making such a contribution within a Unit of Assessment will be able to develop a Code of Practice identifying those staff who fall within and outside this category.
Output numbers to be returned are also identified being a minimum of one output per submitted person within a maximum of five, and an average of 2.5 outputs per submitted full-time equivalent (FTE) staff member. In addition, a sensible and lowest burden decision to the transitional approach on portability of outputs has been adopted where both the institution gaining a member of staff and the one losing the person, will both be able to submit the output assuming that in the latter case the staff member was employed when the output was made publicly available.
Impact decisions
Initial REF 2021 decisions in September 2017 indicated an increase in the importance of research impact raising it from 20 per cent to 25 per cent. The further REF decisions on staff and outputs also addressed the number of research impact case studies required to be submitted which has direct bearing on the significant increase in the importance of impact. In a change to the previous REF, as a result of the decision that all staff with significant responsibility for research be submitted in REF 2021, it has been estimated that the number of staff to be returned would increase by 60 per cent.
As a consequence, the submission rules for impact case studies have been modified in relation to the FTE staff required to be returned per impact case study to maintain the total number of case studies at around the 7,000 assessed in REF 2014. However, unlike this previous exercise the approximate equality in the submitted staff FTE per impact case study required has been broken, potentially changing the overall distribution of case studies associated with discipline areas.
The REF 2021 approach for the number of case studies to be submitted requires a minimum of two for up to 15 FTE staff returned (or up to 7.5 FTE staff per case study), or the same as previously but then 15 FTE staff per case study to 105 FTE staff – that being 50 per cent higher than the 10 FTE staff specified for REF 2014. Surprisingly, however, above 105 FTE staff returned only one additional case study is needed per 50 FTE staff submitted.
A major consequence of this decision is that the smallest submissions will all require an impact case study per 7.5 FTE staff or less returned, while for large submissions the staff FTE submitted per case study will be considerably higher. For instance, a submission of 300 FTE staff will necessitate just 12 impact case studies (25 FTE staff per case study) whereas in REF 2014 that number was 31 (9.7 FTE per case study).
Why does this matter?
Although the fairness – in an assessment context – of the change in the number of case studies per FTE staff returned is clearly open to question, this change will also alter the distribution of case studies between discipline areas. In REF 2014 the total of all submissions in Medicine, Health and Life Sciences (Main Panel A) and Physical Sciences, Engineering and Mathematics (Main Panel B) produced fewer impact case studies than the total of all submissions in the Social Sciences (Main Panel C) and the Arts and Humanities (Main Panel D). The changes to the impact case study numbers required for REF 2021 will result in an increase in this disparity in the overall number of case studies to be returned to Main Panels A and B in comparison with Main Panels C and D.
The disparity can be discerned from a further comparison of sub-panels taken from Main Panel A and Main Panel D; namely the Clinical Medicine and the Art and Design: History, Practice and Theory sub-panels. In 2014 Clinical Medicine had 13 submissions with a staff FTE above 105 (the largest being 450 FTE) from a total of 31 returns whereas the Art and Design sub-panel had just one submission above 105 FTE (that being 110 FTE from the University of the Arts London) from a total of 84 returns. In addition, in the Art and Design sub-panel an increase of 60% or even more in the staff FTE per submission would not add any further to the number of returns larger than 105 FTE. This clearly indicates the distribution of case studies between subject disciplines within Main Panels will change from those submitted in REF 2014 – so that in the overall envelope of around 7,000 there will be a decrease in case studies associated with Main Panels A and B necessitating a pro-rata increase in case studies submitted within Main Panels C and D.
It is perplexing that this approach to the alteration in the number of case studies has been decided upon as it will reduce the proportion of case studies submitted from the STEMM (Science, Technology, Engineering, Mathematics and Medicine) disciplines of Main Panels A and B and further increase the fraction of case studies submitted from the non-STEMM disciplines of Main Panels C and D, despite that they previously contributed over half of the case studies returned.
The situation does not align with the declared position regarding the merit of the production, assessment and publication of impact case studies being that they demonstrate the value of university research to both government and the public, particularly in the areas that support economic growth and improvements to health. It is also interesting that this decision was made following the publication of the Industrial Strategy White Paper which specifically makes mention of the change to the incentives and rewards for capturing the value of research by increasing significantly the importance of research impact in the REF – when the number of STEMM-related impact case studies that could provide a more specific linkage to this Strategy will be reduced in REF 2021.
Lost opportunity
The latest REF decisions also addressed very small returns of fewer than 5 FTE staff through their permitted exclusion from submission in exceptional circumstances. This decision is intended to recognise the concerns raised about such returns meeting the submission requirements despite them containing few staff with significant responsibility for research.
An opportunity has been lost, however, in adopting this exclusion approach. These very small returns could have been exceptionally allowed but without requiring the submission of any impact case studies, the long time period generally needed for the development of which constitutes the main barrier to submission for such small staff groups. This is disappointing, as when small staff groups are more recently engaged in a research area where the university has not previously submitted a return, it would be developmental for the staff and the institution to submit – in relation to both the research outputs that could have been assigned an increased percentage score to take account of the absence of research impact and also for the research environment.
A more inclusive approach, as suggested, would assist the widening and growth of the research base in universities for the overall benefit of UK research.
