The much awaited Contributions to Knowledge and Understanding (CKU) guidance for REF 2029 is out, and finally higher education institutions know how the next REF will work for the outputs component of the assessment. Or do they?
Two of us have written previously about the so-called portability issue, where if a researcher moves to a new institution, it is the new institution to which the research outputs are credited and potentially future REF-derived funding flows.
We and others have argued that this portability supports the mobility of staff at the beginning of their careers and the mobility of staff that are facing redundancy. We believe that this is an important principle, which should be protected in the design of the current REF. If we believe that the higher education system should nurture talent, then the incentive structure underpinning the REF should align with this principle.
We maintain that the research, its excellence, and the integrity with which it is performed depends upon the people that undertake it. Therefore, we continue to support some degree of portability as per REF 2021, acknowledging that the situation is complex and that this support of individual careers can come at the expense of the decoupling and the emerging focus on institutions. The exceptions delineated around “longform and/or long process outputs” in the CKU guidance are welcome – the devil will lie in the detail.
Who the return represents
Leaving aside portability, the decoupling of outputs from individuals has also resulted in a risk to the diversity of the return, especially in subject areas where the total number of eligible outputs is very high.
In previous REF exercises the rules were such that the number of outputs any one researcher could return to the department/unit’s submission was restricted (four in REF 2014 and five in REF 2021). This restriction ensured that each unit’s return comprised a diversity of authors, a diversity of subdisciplines and diversity of emerging ideas.
We recognise that one could argue the REF is an excellence framework, not a diversity framework. However – like many – we believe that REF also has a role to play in supporting the inclusive research community we all wish to champion. REF is also about a diversity – of approaches, of methodologies, of research areas – research needs diversity to ensure the effective teams are in place to deliver on the research questions. What would the impact be on research strategies if individual units increasingly are dominated by a small number of authors?
How the system plays out
Of course, the lack of restriction on output numbers does not preclude units from creating a diverse return. However, especially in this time of sector-wide financial pressures, those in charge of a submission may feel they have no option other than to select outputs to maximise the unit score and hence future funding.
This unbounded selection process will likely lead to intra-unit discord. Even in an ideal case will result in the focus being on outputs covering a subset of hot topics, or worse, subset of perceived high-quality journals. The unintended consequence of this focus could place undue importance on the large research groups led by previously labelled “research stars”. For large HEIs with large units including several of these “stars”, the unit return might still appear superficially diverse, but the underlying return might be remarkably narrow.
While respecting fully the contribution made by these traditional leaders, we think the health of our research future critically depends upon the championing of the next and diverse generation of researchers and their ideas too. We maintain the limits imposed in previous exercises did this, even if that was not their primary intent.
Some might, for a myriad of reasons, think that our concerns are misplaced. The publication of the guidance suggest that we have not managed to land these important points around diversity and fairness.
However, we are sure that many of those who have these views wish to see a diverse REF return too. If we have not persuaded Research England and the other funding councils to reimpose output limits, we urge them at least to ensure that the data is collected as part of the process such that the impact upon the diversity of this unrestricted return can be monitored and hence that future REF exercises can be appropriately informed. This will then allow DSIT and institutions to consider whether the REF process needs to be adjusted in future.
Our people, their excellence and their diversity, we would argue, matter.
With an unlimited number of outputs (and no portability, so if a “research star” leaves, their outputs do not leave with them) the situation might actually be much worse than the authors describe.
The above article focuses on the diversity of the REF submission, but it is also the diversity of the research done which might be affected. Why would a university invest in research that it will then not submit to REF? A situation might arise where universities give more resources (including research time) to “research stars” and deprive everyone else of it. We could end up with a caste system of on the one hand “research stars” with plenty of resources (including research time) and all the others with nothing (including, no research time).
Glad to hear this decision is getting some well-deserved criticism. Getting rid of portability and the limit on outputs are both bad ideas that actually get rid of two elements of previous REFs that were actually inclusive.
In the last REF not only were the number of outputs from anyone researcher capped at 5 but everyone had to submit at least 1 output. While there were no doubt cases of people being shunted onto teaching-only contracts to exclude them from REF, in my experience this did seem to contain the exclusionary practices that were employed previously. At the places I worked at, there was an emphasis on supporting and raising the quality across the board, rather than just relying on a few ‘stars’.
There’s a tension here between two conceptions of REF. One is REF as an external validation of the decision to allocate large amounts of public (i.e. taxpayer) money to universities, by assessing aggregate outputs and outcomes and demonstrating high quality. The second is REF as a way of creating desirable change in universities, by creating (financial and reputational) incentives to deliver certain types of change that could be (at least in principle) unrelated to outputs and outcomes. Keeping those two aligned (or resolving the tension) will be important for the REF to continue to have credibility in the eyes of those who provide the money (i.e. taxpayers).
This really emphasises the importance of the PCE part of REF. For this to work, careers and diversity need to be a key focus of PCE and the weighting of outputs as part of the overall outcome should be (much?) lower. In some ways I can see that assessing PCE is a more direct way of supporting people in the sector in an ongoing way and actually pushes institutions to invest, rather than careers and diversity supported by quite arbitrary limits related to output submission (which, let’s be honest, were just plugged into algorithms to determine best possible submission – not judging btw!). That all said it’s a big change and quite an ambitious move, the impact (if there is one), will be seen in the medium and long term. I think we can see the growing investment by institutions in research culture. Let’s focus on getting PCE right along with the balance across the whole of the assessment structure.
I don’t get this article. It makes an argument about the risks of unrepresentative REF returns largely based on the removal of output limits but fails to to acknowledge that the REF 2029 ‘Contributions to Knowledge and Understanding’ guidance sets out institutional responsibilities to demonstrate that the submitted research should reflect the diversity of subject areas, practices and outputs. I don’t agree with many of the changes for REF 2029 but, at the very least, it should be acknowledged that the REF 2029 team are trying to address this exact issue.
As a academic who took part in the first ‘research assessment’ and all others until retirement I have witnessed, with sadness, the deleterious effect they have had on academia. Is it not time to demand its demise rather than debate the insiduous protocols? Have they improved the research capability, scope, economic impact and excellence in UK universities? This is the debate to be had with a thorough, objective comparison with our international peers.
Thanks for the interesting article. I actually welcome the removal of portability. REF is supposed to be about assessing an institution – so the idea that an institution can effectively buy in research outputs from elsewhere has always seemed strange to me. In institutions I have worked in we have always tried to make it clear that REF is not about assessing individuals – and individual researchers should not be concerned whether their papers are included in REF or not.
The basic question is who “owns” the research outputs of an academic. Your answer seems to be “the institution”. In a normal employment relationship this would indeed make sense. However, the university-academic employment relationship is not a normal one. For example, in which other employment relationship does the employer expect the employee to obtain funding for their activities?
The university is a collection of individuals/groups (with an enormous bureaucracy on top of it).
In a sporting analogy, the university is a bit like the premier league/FA rather than an individual football club. The relationship between academics and universities is therefore more like that between a football club and the FA, rather than that between a football club and a player (the latter being the analogy that is often made, including on wonkhe).
The balance between the relative importance of institutions and individuals to research quality surely depends heavily on the type of research — from “big science,” on the one hand, to single-investigator-led research, on the other. To give two extreme examples: the organisation and culture at CERN likely have a huge influence on the quality of an article with 200 authors involving the Large Hadron Collider, while, at the other end of the scale, the output of Terry Eagleton likely says little about the research culture of the English Department at Lancaster University. This problem was raised some time ago by Nigel Thrift in an excellent HEPI note, which also issued a related warning about mission creep — a trend that has seen REF 2029 shift away from recognising high-quality research to prescribing how universities should determine their research culture.