This article is more than 4 years old

Why pausing the REF makes sense but axing it doesn’t

Research England has announced a delay to the REF submission deadline. James Wilsdon dives into the details.
This article is more than 4 years old

James Wilsdon is professor of research policy at University College London and executive director of the Research on Research Institute (RoRI)

Yesterday’s announcement that the 2021 Research Excellence Framework (REF) is to be placed on hold “until further notice” is a welcome move by Research England and UKRI, in response to the deepening Covid-19 crisis.

Universities, like every other sector, have been working flat out in recent days to adapt to this unprecedented emergency, with the highest priority being given to ensuring the safety and wellbeing of students and staff, and to moving teaching and learning online. Research activities – unless they relate directly to combatting Covid-19, or can’t be paused for technical or security reasons – are, for now, lower down the to-do list, and research assessment is something we can all live without.

Research England’s decision came in response to mounting calls for a postponement from various quarters, including the University and College Union and Emma Hardy MP, Labour’s shadow minister for higher education. However, a senior source at Research England tells me that the institutional views of universities were more mixed, with some pushing for a pause, and others continuity, on the basis that preparations for 2021 are now so advanced that it would create more work to have to repeat this effort at a later date.

Difficult choices

This delicate balancing act is reflected in Research England’s announcement. The current census date – 31 July 2020 – remains in place, by which point universities need to notify Research England of staff numbers being submitted. This is also seen as one way of shoring up job security, as universities will then know that, whatever financial shocks they face over the next year, there is a quality-related (QR) funding stream in place for current staff over the long term. Significant concerns persist over those on short-term research contracts, who urgently need other guarantees, but this is at least one step towards the goal of providing “continued and meaningful assurance” to staff, which the science minister, Amanda Solloway MP, called for in her letter to the research community on Monday.

Beyond the census date, other aspects of the REF timetable will remain up in the air for several months, until there is a clearer sense of the likely duration of the pandemic. At this point, Research England will consult further with the sector, and agree a new submission deadline “no later than eight months prior to the deadline”, giving HEIs plenty of time to prepare.

This decision inevitably prompts a number questions, including whether deadlines for the publication of research outputs, or the generation of research impacts, will be extended. Sources at Research England say this is still undecided at this point, and will largely depend on the sector’s views when any consultation occurs. The REF 2021 steering group meets (virtually) later today and will explore options in more depth.

Whatever the overall timetable, there is also a specific issue related to the impacts of research on Covid-19 itself, and the myriad ways in which universities and researchers are now mobilising in response to the pandemic – whether virologists, epidemiologists, medics and health researchers, engineers working on ventilator design, or economists, social and behavioural scientists, informing and analysing wider social and economic responses. If we want the next REF, whenever it occurs, to reflect UK research at its finest and most useful, then there should be some way of including such impacts in the next exercise, and not waiting until the next cycle, several years down the road.

Rip it up and start again?

Predictably, there have also been a few calls for the entire REF process to be scrapped, or radically overhauled, rather than delayed. Such reactions are understandable: a crisis of this scale forces us to focus on the fundamentals, and I am sympathetic to arguments that the design and systemic effects of the REF on UK research (both good and bad) should be examined afresh once the current cycle is completed. As my source at Research England stressed last night: “It’s the research that matters, not the REF”.

But there’s a hint of what some have dubbed ‘Coronavirus confirmation syndrome (CCS)’ at play in these arguments. To those who would seize upon the current crisis as an excuse to rip up the REF and start again, I’d offer three points in polite response:

First, the current REF is the product of three years of extensive review and consultation following the 2014 exercise. Whatever we may personally think, the design of the REF does reflect the consensus views of the sector, and universities have invested a lot of time and effort in preparing for it. Delay is sensible, but a radical change of the rules at this late stage would just create a lot of extra work, and would upset as many people as it would please.

Second, the REF now performs a diversity of functions in the UK’s research system, beyond simple allocation of QR funding (it has six purposes, if you accept the argument of the 2016 Stern Review). So arguments for redesign that address only one or two purposes (such as allocation) need to explain how REF’s other purposes can be served by other means. For example, any model which removed the impact element (now 25% of the exercise) would seem more problematic in the wake of the decision to remove Pathways to Impact from the grants system – unless we don’t want to incentivise, record and reward impacts at all?

Third, once we get beyond this crisis, and government reintroduces normal constraints and accountability mechanisms for public spending, the university sector needs to be able to make as strong a case as possible for maintaining public investment in research, and particularly in flexible mechanisms like QR funding (which delivers around £2 billion a year). The REF provides us with that evidence – which will be reinforced by the vital and high-profile contributions that so many in our community are making to the pandemic response.

With so much in flux – and with question marks hanging over the financial sustainability of the HE system, and the affordability of extra investment in research (2.4% of what?), as in much else  – we should be careful what we wish for. The time for reviewing and redesigning the REF will come (indeed, some preparatory work is already underway). And I’m all in favour of throwing the whole debate wide open at that point: let’s revisit purposes, methods, consequences. Let’s learn from what other countries do. And let’s look afresh at what metrics, altmetrics and machine learning techniques, can and can’t do for us.

But junking all of the painstaking work that’s gone into preparing for REF 2021 isn’t the answer. Let’s welcome the pause, finish what we’ve started when the time is right, and revisit these bigger questions after the crisis subsides.

6 responses to “Why pausing the REF makes sense but axing it doesn’t

  1. “including whether deadlines for the publication of research outputs, or the generation of research impacts, will be extended”

    Any significant extension of those deadlines would likely also require revisiting the staff census date – if we end up in a situation where lockdown and semi-lockdown conditions are needed for over a year until a vaccine can be made widely available, and so the submission deadline ends up somewhere in 2022, it would seem odd for the submission then to essentially reflect pre-pandemic university conditions (when the sector might look very different indeed), but moving the publication deadline to 2022 while only allowing submission of publications by staff on the payroll in July 2020 would also be very distorting.

    If the postponement needs to be extended, and disruption to the university sector as a whole is widespread – which, as pointed out in other articles here, now looks very likely – then substantial changes to the REF rules may be needed for fair mitigation and assessment. At that point the distinction between an extended postponement, and a cancellation and restart, may just be branding.

  2. The only part of the submission that is majorly affected is impact. Outputs and environment can be completed remotely: they are based on a 6-7 year cycle which is almost complete. There would be no need to extend further into 2022, and I think claims to the contrary are disingenuous. Impact will need a more creative solution, as certain impacts and corroborating evidence sources are now simply not possible. One obvious idea would be to shrink the size of the impact case study submission across the sector by changing FTE thresholds, allowing HEIs to submit 50% less case studies.

  3. ‘the design of the REF does reflect the consensus views of the sector’ – it may reflect consensus views amongst university managers. You may find a very different consensus if all academic staff were asked!

  4. Axing the REF makes complete sense. In 2008 there were 8 UK Universities in the Top 50 Universities in the world (according to the times) in 2021 that number has fallen to 7.The best universities in the world are so inspite of any REF. This is the biggest con hoisted on the university sector. Universities that can do so are gaming the system by appointing people on fractional contracts purely for their publications, often non-UK academics at god knows on what compensation packages. These people do not contribute to anything, they just sell their publications. I bet this is good use of UK tax payers money and works exactly as intended.

    1. I’m involved in the New Zealand version of the REF exercise (PBRF). What an abomination these audits are. There may have been some benefits in focusing NZ research in the first (and maybe second) round; now, the disadvantages vastly outweigh those benefits. The system promotes whole new levels of gaming and gatekeeping, reducing university research to highly-conformed, low-risk, box-checking prattle. It’s compromising the reputation and integrity of the entire tertiary education sector. I don’t even know adjustments could rescue REF/PBRF—at the very least, this would require a monumental overhaul.

Leave a Reply