REF on interdisciplinary research: It’s good but it’s not right
James Coe is Associate Editor for research and innovation at Wonkhe, and a partner at Counterculture
Tags
Today, the REF Interdisciplinary Research Advisory Panel (IDAP), brand new for REF2021, has published its report on interdisciplinary research and equity in the REF2021 exercise.
As long time REF watchers will remember the Stern review made specific recommendations on the treatment of interdisciplinary research in REF. Referencing the British Academy report, Crossing paths: interdisciplinary institutions, careers, education and applications, which noted that interdisciplinary research has an
essential role in addressing complex problems and research questions posed by global social challenges, as well as the increased rigour it can bring to one’s understanding of one’s own discipline
the Stern Review made a number of recommendations to recognise and reward interdisciplinary research. This included better guidance on interdisciplinary research. Interdisciplinary champions on sub-panels, specific assessment for interdisciplinary activity and additional encouragement for interdisciplinary submissions. The review also recommended that if there continued to be a discrepancy between the total amount of interdisciplinary research taking place, and the total amount submitted to the REF, it may be necessary to introduce weightings to specific outputs which were strongly interdisciplinary.
In this respect the report by the Interdisciplinary Research Advisory Panel makes for interesting reading. In the foreword we learn that there is a renewed confidence in the recognition of interdisciplinary research through REF. One of the principal challenges of research of this nature is how to value the merits of the component parts of each discipline compared to the overall quality of an interdisciplinary submission. It seems, that new guidance, altering the composition of sub-panels, and a greater emphasis by the REF team on interdisciplinarity, led to a more consistent approach in its assessment. It is believed this approach led to a greater recognition of research of this kind, and presumably in future analysis, a greater proportion of research classed as being interdisciplinary.
However, some key issues first raised in the Stern review remain unresolved. One of the ideas in this REF was to use a flag system through which sub-panels would recognise interdisciplinary research. This process didn’t work effectively and ended up with some outputs which were interdisciplinary not being flagged and some which were not being flagged. This abrogates detailed quantitative analysis and means that it is difficult to assesses the strength of interdisciplinary research. An opportunity missed and one the four UK research bodies aim to address using a bibliometric approach.
To the future then. The panel sees merit in maintaining the IDAP in the future. However, the panel acknowledges there is room for improvement in developing a mechanism to identify interdisciplinary research. In a post isolation REF exercise the IDAP recommends more opportunity for networking and learning on the nature of research of this kind. There is no suggestion of giving interdisciplinary research a greater weighting of any sort in future exercises.
A step forward undoubtedly in recognising it takes lots of good ideas across lots of disciplines to solve complex issues. A missed opportunity in quantifying and better valuing that work.
As an institutional REF manager, I am not remotely surprised that the IDR flag failed to work effectively. The sum total of the guidance provided was that it was “a flag to indicate to the [disciplinary] sub-panel if the output embodies interdisciplinary research” if appropriate. In some cases, colleagues argued that for their sub-panel, all research was essentially interdisciplinary and therefore the flag was redundant. Others were risk-averse, concerned that applying the flag would change the way in which the output was assessed, perhaps by reinforcing a panel’s thinking about cross-referring it.
What was needed was a definition of ‘interdisciplinary’ (e.g., the output could reasonably fall under the remit of more than one REF sub-panel), a yes/no requirement rather than flagging yes if appropriate, and assurances that flagging information wouldn’t be shared with the assessors.
This is perhaps the best example of where the problem is not with the REF process, but the highly risk averse approach of institutions. The nervousness of institutions around inter-disciplinary research creates the myth that it is not well-evaluated in the REF and despite trying, Research England are not able to counter this long-standing view. Both in terms of my own research and leading roles on REF at several institutions, I’ve always sought to showcase inter-disciplinary research and this appears to have been highly-rated. Institutions that pride themselves on their inter-disciplinarity need to reflect this in their REF submissions rather than blowing their own trumpets until quietening them in REF.