Along with our co-authors we state that a healthy research system does not need peer review. Instead, it needs several main functions, which we expect peer review to deliver: first, certification of the quality of research outputs, funding proposals, and other objects. Second, formation of scientific knowledge and researcher identities and communities (peer reviewing is a way for researchers to ‘talk’ to each other). Third, allocation mechanisms for scarce resources such as research funding or precious space in prestigious journals. If done correctly, these three functions then lead to legitimisation of research results, funding programmes, and of entire research systems.
It is these functions that the research system needs. Peer review is simply the recognisable catch-all process to deliver them. To do so, peer review requires a number of conditions: there needs to be sufficient capacity. We need to be sure that reviewing is carried out to a high standard. There needs to be a suitable diversity of input into the peer review process to ensure all aspects of the work are properly appraised and to ensure an inclusive and equitable system. Finally, review processes should be as open and transparent as possible, while also protecting fairness, safety and the possibility of critical judgement. With these conditions in place, trust can be sustained in the research system, and in science more broadly.
A healthy, trusted research system doesn’t strictly speaking need peer review as we currently know it; it needs processes to deliver certification, formation, allocation and legitimation.
Peer review is experiencing a profound capacity-crisis with researchers spending increasing portions of their time conducting peer review for publication, grant funding, institutional assessment and internal hiring and promotion processes. This ‘peer review burden’ has recently been supercharged due to AI enabling much more rapid writing of articles and grant applications. There are also long-standing issues around the quality of peer review, including built-in conservatism, reviewers tending to prefer work similar to their own, and difficulty to suitably review increasingly complex inter- and multi-disciplinary research endeavours. On diversity and openness, the ‘black-box’ nature of many peer review processes continues to be a matter of concern.
Promising innovations
There are many promising innovations around peer review: publish-review-curate, two-stage processes, partial randomisation, distributed peer review, open review. Plus the usual efficiency-saving mechanisms like shortening application lengths, reducing reviewer numbers or tightening eligibility criteria. While these are important (and occasionally exciting) tweaks to implement, often with demonstrable benefit, in our report we concluded that this will not be sufficient to solve the challenges we set out in this piece. This is because the current malaise is driven by systemic factors that process-modifications can at best dampen, but never solve: hyper-competition amongst researchers performance-managed on metrics of publications and grant income; AI continuing to exacerbate any incentive structure built around production of written items; historic, structural and habitual inequalities reproduced in the current system.
Funding-side
On the funding-side, we need to phase out person-centred grant competitions and move towards competitive allocation of large-scale strategic funding to research-performing organisations. At least for a substantial part of public research funding, this would mean reducing dependence on constant individualised competition and strengthening collective, institutionally supported research capacity. The resulting paradigm will look less like arbitrary conglomerations of research groups clustered around superstar-academics, and more like Fraunhofer or Howard-Hughes institutes, or initiatives often termed ‘centres of excellence’. Individual researchers are no longer managed as grant-winning and publication-writing machines, but as contributors to a centre’s strategic purpose.
However, strategic funding must not become a mechanism for concentrating resources only in institutions that are already prestigious, wealthy or geographically central. A healthy research landscape also requires distributed capacity, regional institutions, smaller universities, community engaged research, local language scholarship, mission oriented networks and diverse organisational forms across disciplines and countries.
‘Strategic’ should not be understood as ‘led by government priorities’ or as a narrow institutional agenda – there are strategic needs for performing arts and gender studies just as there are for biotechnology – but the case for such strategic needs must become explicit and underpin funding allocation. Reviewing happens at three levels here: first, what does a healthy national research landscape actually need and which centres are best-placed to deliver each need? Second, how should a centre’s available funds be spent to best achieve its mission? Third, how is each individual at the centre contributing to the collective endeavour?
This should not mean subordinating researchers to institutional strategy. Rather, it should mean creating collective conditions in which researchers are less dependent on constant individualised competition and better able to contribute to shared intellectual, societal and cultural purposes.
On the publishing side, we propose moving from standard review of all research outputs towards a threefold system: trust markers for every output, i.e. a set of basic checks for elements such as ethics, integrity, and reproducibility; peer engagement, where researchers openly engage with published outputs continuously, rather than as a gatekeeping device for publication; and high-quality evidence synthesis to enable the scientific community, in consultation with societal stakeholders, to assess and weigh up available scientific knowledge on a particular issue.
The future
This is not a short-term vision. We have allowed ourselves to wonder what things might ideally look like not in one or two years, but in ten or twenty. Getting there will above all require a change of funders’ roles towards becoming system-shapers who lead conversations between science and society to define research priorities and distribute funding accordingly. It will also require safeguards to ensure that such priority setting is plural, democratic and attentive to forms of knowledge that are often marginalised in dominant research systems. It will also involve a deep recalibration of what research-performing institutions look like, how they are managed and, most critically, how they manage and incentivise their staff. These transitions will take time, the old ways may only yield grudgingly, and the role of initiatives like the Coalition for Advancing Research Assessment (CoARA) will be critical to ensure crude performance metrics around publication count and grant income become a thing of the past, allowing institutional culture to flourish.
Above all, we judge that a system in line with our vision can provide those key functions we started with: certification, formation, allocation and legitimation – in a high quality way, with capacity and transparency, leading to renewed trust in the research system. The system we envisage will not just certify individual grants, researchers and articles, but strategic endeavours, institutions, bodies of work and collective efforts. Allocation will not be a quasi-organic process largely reproducing existing structures (and, often, inequities), but will be based on several critical questions too often relegated to the fringes: what does a healthy research landscape need? What problems do we need science to solve and who is best placed to deliver those solutions? What is science for and who gets to ‘own’ science? Which endeavours must be centralised and which ones regionally distributed?
Placing such questions at the heart of how the research landscape is organised will be critical for trust and legitimacy. Especially at a time when the more nefarious elements of our politics increasingly take pot-shots at academia, it is time to evolve, defend and shine.
The new report, The Future of Peer Review: A system-wide perspective, by the Research on Research Institute can be found here
