It was just another email from an academic at another university politely declining to review outputs for our REF preparations. It wasn’t the first I’d received, and it certainly wouldn’t be the last.
But this reply stood out.
“I’m doing a blanket ‘thanks but no thanks’ to these requests”, it said. “By my count you’re the 19th institution to ask, so I’m saying no to them all rather than having to choose who to help.”
Nineteen! In that moment, I imagined eighteen of my counterparts, all trying to find external reviewers for their own institution’s internal assessments, all likely disappointed in the same way I was right then. And from that a sense of futility followed.
One more time
Because with so many of us looking to get outputs reviewed, that often had co‑authors at other UK institutions, I knew we could all very well be asking for reviews of the exact same work, over and over again. And that was even before I factored in that most of those research outputs would have already been peer reviewed at least once, before being published either as journal articles or books (other output types are, of course, available!), or that the primary reason we were looking to have them reviewed again now was to help decide whether my university eventually would want to submit them to be reviewed yet one more time by a REF sub‑panel.
The sheer volume of duplicated effort left me stunned, not least as I was experiencing firsthand how this kind of work often meant managing review volumes that would rival some international journals, with institutional systems stretched to their limits, and the gaps patched with little more than Excel spreadsheets and time carved out of already overloaded roles.
I spent that evening cycling between disbelief that this couldn’t really be how things were “supposed to work”, concern that our efforts were only adding to an already very large problem, and a lingering hope that there must be a better, less wasteful way of doing things. While it became clear that there wasn’t anything I could do to change things immediately, part of me stayed wondering how things could be different.
Less work, more use
Setting aside the more drastic options (which may only introduce a whole new set of problems), the potential solutions seem to boil down to two things: we need to review less, and we need to make the reviews we do carry out more useful.
Reducing the number of outputs that are reviewed for REF preparations in the current system feels difficult. Even with specific guidance on assessment criteria, it’s hard to define what makes a given output “high quality”, and peer review remains the gold standard for doing this. The more feasible opportunity, therefore, lies in reducing the overall number of reviews we need to carry out by reusing them wherever we can.
Others have noted the wastefulness and ineffectiveness of how most publishers organise peer review, and I have argued previously that this is partly because, once written, those reviews rarely see the light of day, which limits their usefulness for any other purposes. That said, even if the number of publishers making their peer reviews openly available continues to grow, journal peer review and reviews for REF serve different purposes. As such, we should be wary of suggestions that those reviews could replace our internal reviews or REF sub-panel reviews, not least because doing so would give publishers even more influence when, as a sector, we would generally agree that we want them to have less. And while there are good reasons for journal peer review to evolve beyond mere gatekeeping, we should not expect publishing practices to pivot in service of the REF, which, despite sometimes feeling all-consuming, remains a UK-specific exercise rather than a global one.
We should thus instead aim to make the reviews gathered specifically for our REF preparations more reusable. For example, those assessments should not simply disappear into an institutional filing system, when there are authors who could benefit from seeing the feedback they contain. And this applies to all co-authors. Regardless of who nominates an output, everyone who contributed to the research, at any role or career stage, should get to see the reviews. This may be easier when all the authors are at the same institution, of course, but the bigger opportunity lies in sharing across institutions. That’s where we could really cut down on redundant reviewing across the sector and save everyone both time and money.
Are you now or have you ever?
This could begin with us all simply asking those nominating outputs for review to check with their UK co-authors whether the work has already been reviewed at their institution before commissioning another. This elegant and practical solution was put forward by one of our Unit of Assessment Leads at Loughborough, who had independently recognised the unnecessary duplication of effort created when outputs were being reviewed separately across co‑authors’ institutions. We are now jointly exploring how many reviews we might be able to source via this method as a pilot within that Unit of Assessment, as well as taking steps more widely at Loughborough to ensure that the reviews we conduct or commission will be shareable with all co-authors and, by extension, their institutions.
If enough higher education institutions were willing, this could move beyond such informal exchange to something systematic like a reciprocal data-sharing model, perhaps enabled through a shared online platform: whereby you contribute the reviews your institution undertakes and gain access to those shared by others. The principle is hardly radical. We have already seen it in initiatives such as Snowball Metrics, where institutions submitted their research‑management data in return for access to sector‑wide benchmarking. Applied to reviews for REF preparations, the same logic could reduce the duplication of effort that is unnecessarily straining the system. This would free up institutional resource and the time of academic and professional services colleagues to be re-invested in activities that actually strengthen research itself, rather than just repeatedly assessing it (that said, peer reviewing would still likely be among those activities as it is valuable for authors and reviewers alike (as a way to think, reflect and stay current), but these benefits don’t need it to be so overwhelmingly motivated by REF priorities).
Rinse and repeat
For this sharing model to work, at least two conditions would need to be met. Authors, and more crucially reviewers, would need to give clear consent and understand how the reviews might be reused. There would also need to be sufficient consistency in how reviews are conducted to reassure different institutions that assessments carried out elsewhere are comparable enough to inform their decisions (though the REF’s defined criteria may make this somewhat easier).
This will likely require some of us to rethink how we approach internal reviews, and change is rarely comfortable. But it matters now more than ever. Setting aside the growing issue of reviewer fatigue, financial pressures across the sector mean we are all having to make do with less. On top of that, uneven institutional resources already give some universities an edge in REF, not because their research is necessarily stronger, but simply because they can identify their “best” outputs more effectively, while those without that capacity must just make do and risk weaker performance and reduced funding as a result.
If we fail to address this now and continue to treat peer review as something that must be endlessly repeated, institution by institution, the burden will only escalate, fatigue and quality will only worsen, and the sector will become even more stratified. There is also a risk that, in seeking shortcuts to ease the strain, untested or harmful approaches will be adopted, such as outsourcing review to AI before we have considered if and how this can be done responsibly.
If we, however, recognise that peer reviewers’ time and attention may be scarce but that their reviews shouldn’t be, and instead treat them as a shared asset for the sector, we can start to improve things for everyone. If we are serious about reducing burden, protecting research time and making better use of the expertise in the system, planning our assessments so that the reviews can be shared, responsibly and consensually, is a practical and potentially long-overdue place to start.
Acknowledgements
With thanks to Elizabeth Gadd, Grace Murkett, Jennifer Johnson and Dan Parsons for their feedback on earlier drafts and their encouragement to pursue the ideas within. My thanks also to Richard Giulianotti for conversations about the practical solution he had independently identified, and for his willingness to pilot that approach.
