Since the first Research Assessment Exercise (RAE) was introduced in 1986, such endeavours have always attracted their fair share of critics.
The Association of University Teachers (a precursor to UCU) argued in 2008 that “the RAE has had a disastrous impact on the UK higher education system, leading to the closure of departments with strong research profiles and healthy student recruitment.” Other commentators claimed the exercise placed undue pressure on academics to publish or perish, created unhealthy competition, nurtured gaming by universities and encouraged individuals to go to press with half-baked work – or in some cases, work that was based on falsified data.
And yet, over the same period, the reputation of UK universities for producing brilliant research soared. Britain may no longer be the workshop of the world; its standing as a global political superpower is a distant memory and its economy may have been overtaken by those of other nations; but overall, the UK performs extremely well in global university league tables, only surpassed by the endowment-rich cream of the US private sector.
By other indicators, too – publications, citations, attractiveness for international research talent, patents and reputation surveys – the UK’s most research-intensive universities are recognised as amongst the very best. The foundations of their success are surely based on the excellence of their research outputs and their quality is underpinned by the robust system of peer review which has run continuously since 1986.
The next Research Excellence Framework will cover the period 2021–2027 but the rules of the game are only now emerging. A consultation document issued jointly by Research England, the Scottish and Welsh funding councils and the Northern Irish Department for the Economy indicates a desire to “change the emphasis of national assessment from the performance of individuals to the contribution institutions and disciplines make to healthy, dynamic and inclusive research environments.”
Significantly, the primary focus is no longer on the quality of research outputs – it now encompasses “the broader contributions to research and research processes.” This is intended to be a more inclusive exercise, giving “appropriate recognition to the people, culture and environments that underpin a vibrant and sustainable UK research system”, and “capturing the valuable contributions or a wider range research and research-enabling staff.” To this end, the previous components of outputs (60 per cent), impact (25 per cent) and environment (15 per cent) are replaced by “contribution to knowledge and understanding” (50 per cent), “engagement and impact” (25 per cent) and “people, culture and environment” (25 per cent).
There are several potential issues with this. Most importantly, it moves away from a focused assessment of research outputs, with its weighting now less than 50 per cent, placing greater stress on more nebulous considerations of environment and culture that, by their very nature, are difficult to measure. This represents a significant shift for the REF.
Without question, the research ecosystem has its faults and there is data to illustrate that components of it, such as the funding system, are not equitable. We need to create a fair and inclusive research environment, and to support, reward and celebrate all the varied contributions to research in the UK. But is the REF the most appropriate or best mechanism to achieve this?
Protect the environment
All institutions strive to foster a culture which produces fantastic, world-beating research outputs – drugs that cure diseases, new policies that address societal problems, strategies to solve the climate crisis and dramatic insights into the great diversity of human cultures. These are the goals of research at every institution, and it is the quality of this work that should be measured and assessed.
Instead, we are increasing the assessment inputs rather than outputs. It is a little like judging a world-famous artist not by the quality of their paintings or sculptures, but by how attractive their studio is.
The ever-increasing emphasis on environment threatens to undermine the drive for excellence which successive RAEs and REFs have helped to sustain. It represents a gradual dilution of the focus on competitive spirit which has made UK research the envy of the world. If we want to see enhancements in research environment and culture – and we certainly should – perhaps the best way to do this is to judge institutions by their research outputs. The rest will follow.
While these recent changes to the REF are significant, it is surprising that interdisciplinarity has received little attention. As we seek solutions to complex global social, economic, ecological and political challenges, interdisciplinary research is an increasingly important part of our research activities. Is this a missed opportunity? Or is it just too difficult for what is a discipline-centric assessment process?
The new proposals will likely place further weight on narratives rather than being driven by a genuine attempt at peer review of quality. It is impossible to measure one environment against another on the basis of a narrative. Moreover, there is a danger that the REF will drive a rather prescriptive and unimaginative approach to enhancing research environments, instead of encouraging institutions to put energy into those areas important to them.
By the same token, the assessment of impact will not be made easier by inviting institutions to produce “an accompanying statement to evidence engagement and impact.” If a university has been successfully engaging, this should be evident from the case studies. We should concentrate on the outputs, not the processes or systems which produce them – and we should try, wherever possible, to assess things that are actually measurable.
Perhaps these criticisms are too harsh. Previous research exercises have seen similar radical changes, such as the introduction of the assessment of impact, that have driven positive change. Subject sub-panels are populated by academics who will be able to navigate this new approach. But their jobs would be easier if we avoid creating a system which can be gamed to excess and maintain a laser-like focus on the things that really matter. The REF consultation period ends on 6 October – there is still time to let your views be known.