Many policy developments in UK higher education over the last decade have been negative: added bureaucracy, strained resources, new contradictory and heightened demands, contempt for the sector’s protestations, and a redirection of energy from pure research in deference to non-academic interests. An aura of foreboding reigns.
We could therefore be forgiven for expecting the worst from the Stern Review into the Research Excellence Framework: just another blow against independent, rigorous, and critical inquiry. And yet, with important caveats, there is a core to the Stern Review that goes entirely against the trend, and so holds out the promise of an improvement to the conduct of academic life. To understand why, we must take the following (truncated) sections of the report together:
67. We suggest that in order to maintain the volume of outputs assessed at a similar level as REF2014 – given the increased number of staff to be submitted in the next REF – a maximum total should be submitted be based upon two outputs on average per submitted full-time equivalent (FTE) individual. In other words, all faculty (N) would be returned to a Unit of Assessment, but institutions would be required to submit 2*N outputs for that unit, and would be able to submit more than two outputs for some individuals (up to a prescribed maximum, say six) and less, (a prescribed minimum, potentially none), for others.
…70. The total number of outputs per Unit of Assessment should be adjusted so that total number of outputs to be assessed in the next REF should not significantly exceed the 190,000 reviewed in REF2014. This may require the average number of outputs submitted per faculty member to be below 2, depending on the number of research active staff to be submitted…
…73. We therefore recommend that outputs should be submitted only by the institution where the output was demonstrably generated…
Welcome to the University of Stern
Consider the new regime alongside the old.
At the hypothetical University of Stern, there is a Politics department, and it hires three new staff: Niccolò, Hannah and Gayatri. Over the next 4 years, Niccolò is unfortunately not productive: he has a high teaching load and struggles with adapting his PhD into a book. Hannah, on the other hand, flourishes according to traditional academic measures, and publishes a series of well-received articles and two university press monographs. Gayatri is doing OK in REF terms, producing three solid pieces, but devotes much of her time to engagement with a strong network of activists against police violence.
In the REF-as-is (following the structure of REF2014), the University of Stern should invest in keeping Hannah, to prevent her leaving to go to a more prestigious institution along with her portfolio of 4* work. An early promotion to Professor of Political Theory will do it, locking down Hannah’s outputs for the coming REF.
Niccolò, on the other hand is a sunk cost to the University. If unforgiving managers with pressure bearing down on them happen to be in place, he will likely be encouraged to move to a more teaching-focused role (so that the damage he does to the department’s portfolio might be contained) and perhaps squeezed out of contracted research altogether. If Niccolò has not yet passed probation, there will be significant pressure to speed up his writing (longer hours, sacrificed weekends, poor teaching preparation, a worse manuscript) in order to meet the quota.
As for Gayatri, there is reason for a head of department to apply strong pressure to get one more article out before the REF cut-off date and meet the four outputs required; her collaboration with activists be damned.
In the REF-to-come (Stern’s proposal for REF2021), the problems associated with these three scenarios – the superstar transfer market, the difficult first album, the engaged academic – will each be ameliorated. Since past research excellence is a relatively good guide to future research performance, Hannah will still attract significant interest. She may yet chose to move somewhere more prestigious, but the University of Stern does not have to keep her in order to do well in the REF in the same way as it did in 2014.
There is also a new incentive to give Niccolò relief from teaching in order to produce more outputs, since one good piece from him will still have value. Niccolò may conclude that a different career path is needed in any case, but the reason will not be failure to meet a REF quota, and there is no inherent reason for managers to punish him.
From the perspective of the department, Gayatri is now more readily seen as a REF “asset” thanks to her activist work, and falling below the prior REF quota doesn’t hurt her or the University of Stern at all.
Because the University of Stern can’t depend on the last-minute transfer market, it must also invest in research excellence up front. During REF2014, relatively wealthy institutions could cut their losses and hire in the portfolios, quotas and star ratings as desired. Although the University of Stern had done what it could to foster a positive research culture, it ended up losing several promising staff in just this way. Now it will be rewarded for its support to current researchers, fellows and doctoral students.
Because the average number of publications needed per researcher per REF cycle has been reduced to two (or below), there can still be variation amongst researchers in the number of outputs submitted without putting anyone is under more publication pressure than they were for REF2014. Of course, the University of Stern may respond to the generalised stress on higher education by further dividing research from teaching staff. But the new flexibility around outputs means it has less reason to do so.
Pressure relieved
As matters stand, there is a set model of what REF research competence looks like: roughly one good output a year per research-active FTE. Academics regularly and rightly bemoan this standard for the crassness with which it measures the real life of the mind. Recognising a diversity in publications over a REF cycle, reducing the per-researcher publishing expectation and mandating that all research staff are returned portends for a more open research culture, and a more representative picture of the overall UK research environment.
Individual differences in the number of pieces returned to the REF may produce some bad feeling in academic departments, but what reason is there to think that this will be greater than under the current quota system? Yes, academics work on articles in their spare time, but why is rewarding the university where the research was conducted worse than paying the university that paid your last salary cheque, regardless of whether they contributed anything to the research or not.
The REF is still an evaluation framework, and therefore the assembled criticisms of metrics and audit, their openness to gaming and misuse, their consequences for careers and well-being, their underlying philosophy of ‘quality’ and ‘excellence’, all still apply.
For precarious researchers, there is a new major concern: ‘portable outputs’. The ability for an institution that hires you to claim all your publications in the cycle – is for many crucial in securing a permanent position. The recommendations could even introduce an incentive for universities to wring short-term researchers dry for publications, and then cast them loose. Although this has in some quarters been overstated (universities hire for more reasons than the REF, and countries without a REF still hire early career researchers), there is indeed a significant flaw in the logic of the Stern Review, and it demands urgent redress. There are reasons to think that non-portability will help early career researchers in some ways (since it will lessen the imperative to have publications before getting the first job), but the openness to abuse is an understandable spring for anxiety. One easy fix would be to mandate that the research of anyone on an insecure contract at some specified cut-off date is portable, but for anyone on a permanent contract is not, and this may be addressed in the upcoming HEFCE technical consultation.
In the fracas over early career researchers and output “ownership”, some critics have apparently missed that Stern is recommending that academics publish less. Assuming a constant in REF-related funding (a big if), the implication is a concomitant reduction in the pressure to publish, and consequently a higher quality of research and engagement. We can only hope that this incremental promise is not squandered, either by bodging its implementation, the new bureaucracy of the TEF, or a quid-pro-quo cut in research funding looming on the horizon.
Your hypothetical University of Stern is only one possible hypothetical University of Stern. Consider the following hypothetical University of Stern instead. Hannah and Gayatri both produce 4 outputs which are deemed 3* REF outputs. Niccolò produces 2 such outputs. Hannah and Niccolò are favourites of university/department management, Gayatri is not. Under REF 2014 rules, Hannah and Gayatri would be submitted for REF and Niccolò would not be. Under Stern rules, the department can only submit 6 outputs and those are chosen to be those of Hannah and Niccolò (there is nothing between the outputs, so the outputs of the favourites are chosen). Hence under Stern rules, favouritism rules.
Gayatri used to have value to the university/department because of her quality outputs. Because the university/department has more than the minimum quality outputs needed, she loses this value completely under Stern rules.
Which University of Stern is more likely depends on your view of the current system. If you think that a quota of four pieces rewards people who have done genuinely good work, and prevents managers playing favourites, indeed protects them (the number of good colleagues not being able to meet the quota being negligible), you will see little wrong in the REF continuing as is. It strikes me that this has hitherto been a rather minority position. If, on the other hand, you think that quotas are blunt instruments often already used to apply pressure, and that harm is done to good colleagues for essentially arbitrary reasons related to the quota, you will see value in both reducing the pressure to publish and accepting greater variation in which pieces are chosen. Which means supporting at least some aspects of Stern’s review.
Those who claim that submitting more pieces makes you the better scholar in the proposed new system have in my view drunk a little much of the existing REF Kool-Aid. It is a view that reflects the emphasis on meeting a quota. Stern advocates a different emphasis. For the alternate scenario to be widespread departments would have to be (a) swimming in high quality submissions; (b) defined by a particularly cruel management class. I tend to think (a) unlikely in any but a few places, and (b) more a result of perverse incentives in the system than a question of widespread malice.
In the alternate scenario, Gayatri is ‘valueless’ not because the system’s requirements decree it, but because her boss doesn’t like her. It is always possible to postulate that our betters will act with arbitrary cruelness, but this is not something that a systematic review can legislate for. It can only be expected to address plausible structural incentives.