Deliver us from rankers

It feels churlish to respond with a weary snarl to the launch last week of THE’s University Impact Rankings. Sustainable development, wellbeing, decent work, gender equality – what’s not to like?

And of course, at one level, anything that incentivises, recognises and rewards universities for their diverse and myriad contributions to society is a good thing. As Jonathan Grant of King’s College London describes, participating in the pilot round of this exercise was, for them at least, a positive experience, which “challenged us to think even more broadly about how we considered our social impact.”

Yet I found myself struggling to applaud this latest beauty parade – or to take seriously the proposition that Auckland is top of the impact pops, and Canada is the best-performing nation. This has nothing to do with my own institution’s – eminently sensible – decision not to take part. Indeed, my reservations about university league tables are both evidence-based and longstanding.

But there’s something about the “impact rankings” that sticks in the craw.

Rankings: Infinity Wars

In part, it is the sheer boredom at the proliferation of increasingly meaningless rankings now being produced on a weekly basis. THE is the worst offender by far – I would list all their rankings here, but it would quickly exceed the word-count of this piece. We all recognise that, as THE itself acknowledges, it has “transitioned from being a primarily UK-focused media company” into a private-equity fuelled “global data and insights business”. But perhaps new owners Inflexion, who acquired THE last month, might recognise that, just occasionally, less can be more, and there is a fast-diminishing return – in attention, if not revenue – from bombarding us with a new league table once a week.

THE reminds me of Marvel – wringing every last twist from its rankings franchise, while its audience yawns and switches off – or prays silently for a higher education equivalent of Thanos to wipe out half of all rankings in the known universe.

More seriously, if THE really wants to encourage a stronger focus on the social, environmental and economic impacts of the university sector, by far the easiest way of achieving this would be to incorporate some of the positive indicators in these new rankings – around equality, employment practices, inclusive education, resource consumption – into its main world rankings. At a stroke, this would achieve what many critics of rankings have called for, by diversifying and pluralizing measures of global “excellence”, and spurring far more than the 500 universities which participated in the 2019 impact pilot to give these issues greater priority.

For all of its bureaucracy, this is what the UK’s REF has achieved, by giving impact a serious weighting alongside research outputs.

Methodology, please

Beyond this, a further way in which rankers could make a genuinely progressive contribution to a more diverse, equitable and sustainable higher education sector would be to treat the communities that it ranks like the data-literate grown-ups that they are. Across all league tables, we should see an acknowledgement of the uncertainties in the data, a concession that weightings are arbitrary and open to debate. A stop to organisations cross selling between rankings and consultancy arms would help, too.

At a time when the big four audit firms are facing growing calls for an enforced separation of their audit and consultancy services, we as a sector need to scale up the scrutiny we apply to the rankers, and demand greater transparency and accountability. Criticisms of their methodological flaws and pernicious effects are nothing new.

We restated these arguments four years ago in my Metric Tide review. But while there has been tangible progress by many universities and funders in their commitment to responsible metrics and evaluation methods, the pace of change among the rankers is painfully slow

One of the UN’s sustainable development goals – SDG 16 – encourages greater accountability, scrutiny and transparency of key institutions. Across our sector, in little more than a decade – rankers have become increasingly visible and powerful actors. If they want to prove that they care about sustainability and social impact, it’s time to put their own house in order.

Otherwise, I expect more vocal and creative grassroots efforts to resist rankings. And I for one will be cheering them on.