Universities in Hong Kong make a significant contribution to society. We know this from the release, this week, of a searchable database of impact case studies (ICS), and a series of reports that document the impact of university research in different sectors.
As with the UK REF 2014 and 2021 exercises, this analysis demonstrates the value of assessing research impact through narratives. From a methodological viewpoint impact case studies provide a rich set of data to mine and from a policy perspective their synthesis makes the case for research as well as incentivising and rewarding socially impactful research.
The Hong Kong RAE
The Hong Kong University Grants Council (UGC) is an independent non-statutory body established in 1965 that advises the Government on the funding and strategic development of higher education in Hong Kong. Since 1993, the UGC has conducted six Research Assessment Exercises (RAEs), modelled from the UK system, as a part of its performance-based assessment process. Each RAE assesses the research quality of the eight UGC-funded universities.
In the latest exercise in 2020 there were 41 subject-based Units of Assessment grouped under 13 main panels. RAE 2020 introduced research impact as one of the new assessment elements. Impact was assigned a 15 per cent weighting towards the overall quality profile (the other elements were: research outputs – 70 per cent; and research environment – 15 per cent).
About 16,000 research outputs involving some 4,200 eligible academic staff, 345 ICS and 190 research environment submissions were assessed in the 2020 exercise. Results and published documents regarding the RAE 2020, including the principles and the methodology, and those of the previous RAEs, are accessible from the UGC website.
We were asked by the UGC to develop a database and produce the reports that synthesise the overall impact of research from Hong Kong universities. Not only was this a real privilege, but it was fascinating to compare the findings with the 2014 and 2021 set of ICS from the UK REF.
One of the key advantages of the Hong Kong set is that it was large enough to text mine and small enough to read. (Three ICS were removed from the analysis for confidentiality reasons meaning we examined 342). This meant that in addition to an overarching report we produced 11 reports based on “impact topic clusters” identified through the text mining but comprising a narrative summary of research impacts and their characteristics, based on reading and coding the ICS.
Considerable and diverse
Our take home message was how similar the key observations were to the synthesis of the UK ICS – the impact of university research is considerable and diverse, societal impact is based on multiple research disciplines, the reach of impact is local, regional and international, and that social impact is underpinned by highly cited research.
Some extraordinary examples of impact from the Hong Kong dataset include DJI Drones, a company started by a Hong Kong university Master’s student based on the research undertaken as part of his studies, which in 2017 had US$2.83 billion annual revenues; the annual saving of 25 million kilowatt-hours from optimised design of air conditioning systems and the integration of renewable energy and smart, energy efficient control strategies; and the viewing by 20 million people of a series of televised lectures on East-West cross-cultural understanding and misconceptions.
The pathways to impact, illustrated in the alluvial diagram, demonstrated the multidisciplinary nature of research impact. On the left we trace pathways between research disciplines of underpinning papers and the eleven impact topic clusters and on the right, between topic clusters and the Hong Kong RAE subject panels. The colour bands are determined by the topic clusters. The multiple lines leading into the eleven topic clusters in the middle, illustrate the multidisciplinary nature of the underpinning research, whether organised by research discipline (left hand side) or submitting panel (right hand side).
Mapping the reach of research impact
Another observation was that three quarters of the ICS described benefits that were being realised in Hong Kong. It was possible to map these benefits using the Hong Kong Standard Industrial Classification, illustrating how they concentrate in thematic areas. For example, ICS in the Engineering, Energy & Construction topic cluster had a widespread impact across construction (42 per cent), logistics & transport (28 per cent) and energy (17 per cent), whilst Production & Technology Transfer influenced manufacturing (50 per cent), information and communication services (27 per cent), and health & social sector (27 per cent).
At the same time, around two-thirds of the impact described in the ICS is realised in Asia, predominantly in the Asia-Pacific region. The other two continents gaining a significant impact benefit are Europe and North America and this matches the regional spread of both academic and pharmaceutical/industrial collaborative partnerships identified by the researchers.
Finally, as with the UK ICS we also found that research underpinning social impact is globally outstanding. The average and the median normalised citation scores were calculated for 1,446 papers that were indexed on the Web of Science. The overall average was over four times higher than would be expected. In part this could be a consequence of a general selection of the more highly cited material for inclusion in these ICS, although the overall citation impact of all Hong Kong research was also high (1.8 times what you would expect).
Our overriding impression of undertaking this work is that, yet again, when you go to the effort of amassing a set of case studies that tell the story of how research contributes to society it irrefutably demonstrates the value of research and research funding.
