REF 2028 is not going to look like REF 2021.
The reception so far has been largely positive, but some misgivings about unintended consequences have also emerged. As ever, the devil will be in the detail.
Individuals and their environments
This exercise will finally break the link between individual researchers and REF, completing a movement that has been under way over successive assessment exercises. Institutions will no longer submit individual members of staff, and there will be no census date. There will be no minimum or maximum output requirement, which means that the need for an individual staff circumstances process will vanish. This was a difficult and onerous aspect of REF 2021, and its disappearance will be a relief.
The emphasis on the assessment of a wider research culture, rather than measurements of productivity, reflects a long-overdue acknowledgement that assessment affects behaviour, and that it may be used as a means to influence research environments for the better. This is welcome. But the implications for staff who for a variety of reasons have found their research productivity to be constrained over the course of the assessment cycle will have to be carefully managed. Is there a danger that they will fade to the margins as departments plan their submissions? And will the disappearance of the limit on the number of outputs that may be submitted mean that departments try to employ stars with large numbers of strong outputs to boost their profiles?
The people, culture and environment element of the submission which will be primarily questionnaire-based, will provide the means to mitigate these risks. But it will need to be framed with scrupulous care, if it is to guard against the ever-present risk of game-playing. What proportion of a department’s researchers are employed on short-term contracts? Is there a gender pay gap among staff with significant responsibilities for research? How is the work of groups of staff who have been under-represented in previous exercises supported? What sanctions might be imposed on departments submitting a selection of outputs that are not fully representative of their staff profiles? These are questions that will matter.
Culture and its outputs
The new prominence of research culture is accompanied by a demotion in the importance of the assessment of outputs. A wider variety of research outputs will be eligible, and the range of staff permit to submit work to the REF will now include research-enabling staff (technicians, for example). This is a far-reaching reform, reflecting REF 2028’s broader definition of what research excellence means, and how it should be evidenced. Thoughtfully managed, it could be a wholesome move away from the idea that the value of researchers can be easily quantified by counting the number of their outputs, which would usually take the form of journal articles. The danger is that the shift towards the assessment of research management and culture might seem to diminish the importance of the research itself. Research-intensive institutions have the most to lose here, and they are uneasy about the change. This aspect of the remodelled exercise will call for careful communication.
Impact, now re-named engagement and impact, will continue to represent 25% of the final quality profile. Here too an evidence-based explanatory statement on wider contribution beyond submitted impact case studies (ICS) will be required, constituting at least 20% of this element. This is a sensible move away from the entire dependence on ICSs for the Impact score in REF 2021. Further positive amendments to this element of the assessment are proposed. The requirement for a minimum of two Impact Case Studies (ICS) per submission will be reduced to just one. Small departments, who often struggled with this prerequisite, will rejoice. The rule that the research underpinning an Impact Case Study must be of 2* quality or better, always a tricky criterion to assess, will also disappear, and will be missed by no-one. But uncertainties remain. The contribution of engagement is now specified alongside ‘Impact’, but it is not yet apparent how this will be defined or assessed.
Not everything is yet settled, and further consultations will follow. Some of these will address matters of continuing concern. The alignment of REF with UKRI policy engagement, for instance, will create challenges – not least around Open Access requirements, where a flexible approach will be needed in some fields, including the increasingly important area of Creative Writing. But we can be confident that REF 2028 will recognise the importance of a positive research culture more fully, and will include a wider range of activities, outputs and impact than was previously possible. With thoughtful implementation, it could deliver genuine change for the better. Whether this will finally be the exercise that delivers the long-promised benefit of a reduced administrative burden is another matter, and remains to be seen.
Only requiring one impact case study for small submission is indeed fantastic news.
But I’m still wondering why very large submissions are allowed to submit fewer impact case studies per FTE than small and middle-size submissions – I just don’t see the equity in this… at all. For large submissions each case study will be worth more and they can focus more resource into them.
I’m not sure one ICS will capture the full range of impact, even for ‘small’ submissions, whatever that may look, like and how it will be defined. If it focussed on an extended narrative, that may assuage some of the concerns.
Does this mean staff employed by NHS that contribute to their allied academic institution’s research output will also be included?
There’s nothing positive at all about a ‘research culture’ orientated to the REF rather than actual research. Let’s scrap the whole farce, and just divide up research funds in each subject area equally per researcher.