Rankings influence rankings

Academics, too, can be led astray by rankings

According to a new study reported by the Chronicle of Higher Education, it’s not only prospective students who are swayed by league tables. It seems that academics’ opinions can be distorted by rankings, even when judging a university’s quality in their own subject.

The fault appears to lie with a psychological phenomenon known as the “anchoring effect.” When people are asked to make judgments in ambiguous circumstances—by, for example, putting a dollar value on a home or a used car—most will start with whatever information is made available to them and work from there. Show people a jar of pennies, and ask one group if it contains more or less than 400, and another group if it contains more or less than 800, and the first group will subsequently tend to offer much lower guesses of the actual number of pennies in the jar than the second. Show potential home buyers a house with a relatively high list price, and they will be more focused on its positive features than they would have been if the list price were lower.

And it seems this applies to rankings too. The study examined the Times Higher Education international league table in its first and subsequent years:

In their analysis, Mr. Bastedo and Mr. Bowman found that results of the reputational survey conducted in the second year much more closely matched the overall published rankings than did the results of the first year’s survey. In other words, institutions that ranked high in the first year fared significantly better on the reputational survey in the second, suggesting that the widely publicized rankings had helped a consensus form around the perceived superiority of certain institutions.

And the same applied to subjects too with universities with a modest result in a specific field scoring much better in the same subject in the following year if the overall institutional ranking was strong.

So, it seems that rankings influence rankings. Who’d have thought it.