Very Meta: Ranking Higher Ed Systems

The latest U21 systems ranking

Now an annual event, we have the Universitas 21 league table, the 2017 Ranking of National Higher Education Systems, which is intended to give an overview and ranking of higher education systems across the world. This is the sixth such report and now has some additional features:

The Ranking now also includes additional adjusted results, taking each country’s level of economic development into account as well as measures of productivity of the higher education sector. The 2017 ranking incorporates a new measure of the diversity of institutions, comprising two equally-weighted components, recognizing more fully that a good system of higher education provides a range of institutions to meet differing student and national needs. Further details of these measures can be found in the full report (attached). New for 2017, the U21 Ranking of National Higher Education Systems also includes measures of the three types of activity undertaken by tertiary institutions, referred to as the triple helix.

The full report gives much more information about the rankings (and the ‘triple helix’) but in summary:

The rankings of institutions are essentially rankings of research-intensive universities and as such are encouraging a bias in systems of higher education towards that type of institution. A good system of higher education will encompass a range of institutions. The need for a diverse system is cogently argued by the former tertiary education co-ordinator at the World Bank, Jamil Salmi (2017, p.237):

At the end of the day, the best tertiary educations systems are not those that boast the largest number of highly ranked universities. Governments should worry less about increasing the number of world-class universities and dedicate more efforts to the construction of world-class systems that encompass a wide range of good quality and well-articulated tertiary education institutions with distinctive missions, able to meet collectively the great variety of individual, community and national needs that characterise dynamic economies and healthy societies.

We use 25 measures of performance grouped into four modules: Resources, Environment, Connectivity and Output. The rst two are input measures and the second pair measure outcomes. For each variable, the best-performing country is given a score of 100 and scores for all other countries are expressed as a percentage of this highest score. A description of each variable is given in the relevant section and sources are given in Appendix 1. Our methodology is set out in detail in Williams, de Rassenfosse, Jensen and Marginson (2013).

Resources, whether public or private, are a necessary condition for a well-functioning system of higher education, but they are not sufficient. A well-designed policy environment is needed to ensure that resources are used well. A consensus is emerging that the preferred environment is one where institutions are allowed considerable autonomy tempered by external monitoring and competition. The Environment module contains measures of these characteristics.

Turning to outcomes, our output measures encompass attributes such as participation rates, research performance, the existence of some world class universities, and employability of graduates. There is a world-wide trend for governments to encourage institutions of higher education to strengthen relationships with business and the rest of the community. The Connectivity module includes variables which span this wider concept.

So, it’s not simple and straightforward and all of the detail can be found on the website which also includes, excitingly, an interactive map containing a country-specific summary for each of the 50 countries included in the report.

Looking at the actual rankings we find that the UK does reasonably well but the overall picture is, unsurprisingly, quite stable:

 

The United Kingdom is ranked third overall, which combines ranks of 6 for Resources, 1 for Environment, 2 for Connectivity and 2 for Output.  As a share of GDP, government expenditure on higher education is ranked 21st, total expenditure eighth and research expenditure 19th.  Expenditure per student is ranked fourth which reflects the lower than average participation rate (ranked 39th).  Connectivity with industry is relatively strong:  the United Kingdom ranks ninth for knowledge transfer with business and 13th for joint publications.  In joint publications with international authors it is ranked 15th.  The United Kingdom ranks fourth for the percentage of students who are international and fourth for the number of times external users access websites (deflated by population).  In the Output category, the United Kingdom ranks third for total publications, eighth for publications per head of population and sixth for the average impact of articles.  It ranks second for the quality of its best three universities.   It is ranked ninth for the (tertiary) educational qualifications of the workforce and 16th for researchers per head of population.  UK (England) students and recent graduates are ranked 21st for literacy and numeracy.  When levels of GDP per capita are taken into account, the United Kingdom is ranked second and its score is well above the level expected at its income level.

Aggregating to obtain an overall ranking, the top ten countries are:

 

Overall Rank in 2017 Country Rank in 2016
1 United States of America 1
2 Switzerland 2
3 United Kingdom 4
4 Denmark 3
5 Sweden 5
6 Singapore 8
7 Canada 9
8 Netherlands 7
9 Finland 6
10 Australia 10

 

Using weights of 40 per cent on Output and 20 per cent on each of the other three modules, the top five countries, in order, are the United States, Switzerland, the United Kingdom, Denmark and Sweden.  The only change from the 2016 rankings is that Denmark and the United Kingdom have swapped positions.  The next five countries are Singapore (up two to sixth), Canada (up two to seventh), the Netherlands (down one to eighth), Finland (down three to ninth) and Australia unchanged at tenth.

Systems evolve slowly over time, compared with the 2016 rankings, for 33 of our 50 countries the rank change was at most one.  The largest changes have been Ukraine, up seven places to 35th, and Turkey, up five places to 40th.  The largest fall in rank is Brazil: down four places to 42nd.

There is much more to scrutinise in the report itself.

Finally, it is worth noting the alternative ranking on offer here which is GDP adjusted rank:

As in 2016, this data has also been compared against the values expected at each country’s level of economic development, to create a second and separate set of ranking results:

GDP Adjusted Rank in 2017 Country Rank in 2016
1 Serbia 2
2 United Kingdom 1
3 South Africa 7
4 Denmark =3
5 Sweden =3
6 Finland 6
7 New Zealand =9
8 Portugal 8
9 Israel 11
10 Australia 14

 

Using this adjustment, the top three countries in rank order are Serbia, the United Kingdom and South Africa.  These are followed by three of the Nordic countries: Denmark, Sweden and Finland.  Compared with the original rankings in Section 3, nine countries improve their ranking by at least twelve places.  These countries, in order of the ranking improvement, are Serbia, South Africa, India, Portugal, China, Brazil, Ukraine, Croatia and Greece.   In several of these countries real income growth has been low or negative in which case some stickiness in higher education performance will translate into an improvement in the GDP-adjusted rankings.

All very meta. Much to chew on.

0 thoughts on “Very Meta: Ranking Higher Ed Systems”

Leave a Reply