Five REF myths to watch out for
Jonathan Grant is director of Different Angles and a contributing editor of Wonkhe
Tags
REF is expensive. Not so – previous iterations cost about 2-3 per cent of the total value of QR funding versus around 10-15 per cent for project grant funding. It feels expensive as it is concentrated into a relatively short period. If you really want to be “efficient” in the allocation of performance related research funding you would wind-up the research councils and allocate all UKRI’s funding via QR and REF.
Metrics provide a cheap alternative. Typically a political response to the perceived costs, but this only works if you move away from the unwritten principle of REF (and previous RAEs) that every discipline should be treated the same. It might be possible to use, say, normalised citation counts from some of the Panel A (medical and biological sciences) units of assessment, but simply not feasible for those in Panel D (humanities and creative and performing arts).
Academics hate REF. This is not to say that they love it, but to say that after every iteration of REF or RAE there is a reflection period that looks at alternative approaches. I recall vividly doing a project for then HEFCE as part of the Roberts Review in 2003 going around the country running design workshops for an alternative approach and always ending with the preferred model being a REF-like review – that is academic-led with peer review at its heart.
Research assessment is not needed. Unrealistic. Research is largely funded by taxpayers and as a sector we have a responsibility to demonstrate what we do with their hard earned cash. The idea that research should be exempt from any form of accountability is wrong and feeds an entitlement culture that further undermines the social contract that lies between universities and wider society.
The quality of [fill in blank] research has improved. Research England and every university will have spent last week drafting press releases claiming that UK research – and their university research – is “world leading” and has improved its performance since 2014. Be cautious in calibrating the results of REF against external benchmarks and over time. In the 2014 REF, despite claims that the UK had doubled “world-leading” (4*) research in Life Sciences and Medicine (Panel A), we showed with bibliometrics ‘only’ a 10-25 per cent improvement in top cited papers.
Great to be reminded of these basics. Much of this week’s froth will be aimed within the sector, but with everybody shouting, little will be heard.
Would be interested to hear what is included in cost here -and how it differs across HEIs
Good question. There is a veritable industry of internal ‘shadow REF’ activities designed to game the system and get the best reputational results from it, sitting alongside highly paid senior institutional managers appraisining ‘strategic’ advantages of the process. These are not attributed to the official costs of the exercise. One has to ask not only the full economic cost of this internal activity is, but what the opportunity cost of paying for this against other potential institutional priorities like supporting continuing and adult education, mounts up to. If only we had expert panels of academics in economics and education disciplines who could work together to evaluate this …
I note the idea that the REF is norm-based – promoting a particular culture of disciipline-based research and discriminating against novel or post-discipline approaches – is not regarded as a myth.
One only has to look at what is favoured in the economics discipline to see this – at the time of the global financial crash, shortly after 2008, someone did ask how much of the output of those focussing on econometric based approaches favoured and rewarded by the REF-type methodology actually forewarned us of the inevitability of the crisis.
I think this raises the question mark over how much the large amounts of public monies disbursed through the longstanding QR funding mechanism benefits those people who pay for it. At one time the QR funding for Theology was more than that for Social Work … as one prominent VC said at the time, the idea from universities own activities that research into supernatural explanations is going to be more useful at tackling social problems than looking at them directly is somewhat questionable.