Everything you need to know about REF 2029

REF 2029 has been unpaused and with it will undoubtedly come a whole new wave of disagreement and debate. Much like the research ecosystem itself it is an unpredictable beast forever buffeted by its participants, leaders, and funders.

To get immediately to the headlines. People, Culture, and Environment (PCE) has been relabelled as Strategy, People and Research Environment (SRPE). The weighting for the new element is 20 per cent of the total dropped from the 25 per cent weighting originally allocated to PCE. Contribution to Knowledge and Understanding (CKU) (the output one), has been boosted to 55 per cent, and Engagement and Impact (E&I) (the impact case study one) has remained at 25 per cent.

There is a significant attempt to reduce the burden of the exercise through reverting to some of the narrative practices of REF 2021 in research environments while reducing the need for new data to be collected. E&I has remained pretty much the same and there are some concessions on portability that will only partially assuage the concerns of people concerned by this sort of thing.

So: a bit more for the things that researchers produce and a bit less for how they produce them.

Strategy people research and environment

The big frame for REF 2029 has been that research is a team sport. This is why Research England and its devolved counterparts have sought to remove the relationship between researchers and research outputs. However, this led to a forever debate on who actually produces research between institutions or researches and whose work should be measured. In effect, should an average researcher be boosted by an exceptional research environment or should an exceptional researcher be held back by an average research environment.

SPRE asks institutions and units to demonstrate how their strategies contribute toward the development of people and good research environments. This will be done primarily through narrative with metrics to support. There is flexibility in how providers may demonstrate their work in this area but the core idea is that the work should be accompanied by a clear strategic intent.

The actual basket of work that can fit under SPRE is varied and might include evidence of improving research cultures, new partnerships, collaborations, the development of new policies, and a range of evidence of improving culture metrics. The major change from what had been proposed is the underpinning focus on strategy and by extension the broader range of activity providers are likely to submit. Culture is still very much there but it is part of a range of activity.

SPRE will be assessed at both an institution and unit level. The assessment will be through a statement similar to the unit level statements from the environment element in the 2021 exercise. However, the institution level score will make up 60 per cent of the overall score for each Unit of Assessment (UoA) and documentation linked to the UoA itself will make up the remaining 40 per cent of the overall score. In effect, this means that the research infrastructure of the institution will have a greater impact than the research infrastructure of the unit where research is actually produced.

The changes to SPRE have partially emerged from the PCE pilots. Their conclusion was that it would be possible to assess PCE, but that the approach would need some adaptations for a full scale exercise. Some of the challenges included: the phenomenon of larger institutions scoring better purely because they had access to more evidence, the need for simple and timely data collection, and a need for clearer guidance and simpler processes. In short, it is technically possible to measure PCE in a robust way but it is hard to implement – which was a view shared by many at the start of the exercise.

Measures

The argument in favour of the 60:40 split is that it incentivises providers to improve their research environments across the whole institution. In what will be partially good news to the minister there is also a renewed focus on rewarding providers that are focussed on aligning their activities with their strategic intent in people, research, and environments.

While we do not yet have all of the criteria, the submission burden seems to be lower than many feared. As well as the statements at a unit and institution level there will be a data requirement at an institutional level which it is anticipated may include: which units are submitted, volume, research doctoral degrees awards, and annual research income by source.

At a unit level there are similar set of measures with some nuance. In the initial set of decisions it was proposed then PCE now SPRE could include

[…]EDI data (that are already collected via the HESA staff record), quantitative or qualitative information on the career progression and paths of current and former research staff, outcomes of staff surveys, data around open research practices, and qualitative information on approaches to improve research robustness and reproducibility.

There are criteria yet to be published but it is suggested that issues of equality will be looked at primarily through the statements, and through calibration with the People and Diversity Advisory Panel and the Research Diversity Advisory Panel during the panel assessment stage. The data burden will be less and ideally not newly collected.

CKU OK

SPRE will also now be the place where institutions submit context, structure, and strategy, about their units. Disciplinary statements have been removed entirely from Contribution to Knowledge and Understanding (CKU) and Engagement and Impact (E&I). This might look like a rearranging of the same information but it also impacts the overall weightings.

In the previous model CKU accounted for 50 per cent overall including outputs and the statement. In effect, CKU now accounts for 55 per cent of the weighting while focusing only on outputs.

REF is now an exercise which is still majority related to the perceived quality of research outputs. There is now an upper limit on individual submissions of five per unit unless there is an explanation why it exceeds this. However, there is no requirement that every researcher submits (the decoupling process). Providers will have to produce a statement on how their submissions are representative through and each unit will be expected to provide an overview of their work and a statement of representation.

On the other big debate the portability rules have remained broadly the same. To recap, in REF 2014 the whole output was captured by whichever institution a researcher was at, at the REF census date. In REF 2021 if a researcher moved between institutions the output was captured by both. In REF 2029 the initial proposal was that the output will be captured by the institution where there is a “substantive link.” Research England has made a slight concession and will allow long-form submissions to be portable for a five-year period with sufficient justification.

What remains unresolved

There is a political element to all of this, of course. In the post-16 white paper it was made explicit that

We anticipate that institutions will be recognised and rewarded, including through the Research Excellence Framework (REF) and Quality Related funding, for demonstrating clarity of purpose, demonstrating alignment with government priorities, and for measurable impact, where appropriate. While government will continue to invest across the full spectrum of research, we expect universities to be explicit about their contributions and to use this framework to guide strategic decisions.

REF 2029, as currently set out, does not do this. Unless there are further announcements on the relationship between REF and funding, REF will do a different version of what it has always done. It assesses the research that is put in front of it. There is no additional weighting for alignment to government priorities, there are no changes to impact measurements, and while there is a focus on alignment between activity and strategic intent it is up for institutions to define what that strategic intent is. There have been efforts to reduce the burden from the initial decisions but this does not seem to be a significantly less burdensome activity than REF 2021.

The minister might be pleased that the word “strategy” has replaced “culture”, and with some fiddling with weighting, but the direction of travel across the whole exercise has remained broadly intact. It is not quite the cultural revolution that was promised nor is it an output focussed exercise that some wanted. It’s a bit of a compromise but only a little bit.

What now

The response of the sector will largely determine whether these changes are viewed as a success. Ultimately, REF is a political project. It is not simply an input into the dispassionate allocation of public money but requires decisions on what is valued. There is a version of the REF which is only about research outputs. There is a possible version which is only about research environments and there are hundreds of weighting, criteria, frameworks, rules, and regulations in between.

The reasonable criticism of Research England is that it made radical changes to REF 2021 and could not bring the sector with it for REF 2029. At times, it felt like the public explanation was about how a series of technical changes to the exercise achieved a set of good outcomes for the sector without vigorously explaining what good was, who would lose out, and why the trade offs were worth it.

These new decisions are either a messy middle ground or a genius compromise. They cede ground to those concerned about outputs by changing weightings and moving criteria but it maintains culture as a key focus. They provide room to include more culture focussed statements without complex metrics. And they are politically astute enough to talk about strategy, even if the strategy isn’t the same as the government’s in every institution.

The worst possible result would be the ongoing argument between providers, between providers and funders, and between funders and government. The unedifying spectacle of a noisy debate on why elements of the sector’s own research exercise is not fit for purpose distracts from both the enormous administrative burden of the exercise and the political case for why the sector should command significant research funding.