Evidence for Enhancement: the Scottish approach

Last week the Scottish HE sector came together under the auspices of QAA Scotland to launch its new Enhancement Theme ‘Evidence for Enhancement: Improving the Student Experience’. On such occasions it feels like a real privilege to work in the Scottish sector with its unique approach. While the sector is grappling with the challenge of preserving a UK-wide approach to quality, such as the proposed new Quality Code, it is timely to celebrate the strengths of the collegiate Scottish approach, in the hope that the whole sector can learn from our experience.  

This is the eleventh Theme since 2003, with the early heady days of two Themes a year having long since given way to an approach which sees the whole sector working together on one theme over a three year period. The Enhancement Themes are a key element of the distinctive Scottish approach to quality enhancement, and have been instrumental in the development of what is now a deeply rooted culture of collaboration across the whole sector. Each of Scotland’s 19 higher education institutions brought a team of student representatives and staff, to work alongside colleagues from sector agencies including NUS Scotland, the Higher Education Academy, sparqs, the Scottish Funding Council and Scottish Government. Together the group shaped the direction of the Theme, which felt particularly timely given a challenging context for so many students and institutions.

Using (and abusing?) data

Followers of Wonkhe will be well aware of the coming revolution in the availability of data and evidence in higher education, with all the benefits it can bring to students and staff. However, be it externally generated (e.g. TEF, LEO) or internally produced through increasingly sophisticated systems that underpin learning analytics, such data clearly also has the potential to be used and abused to drive agendas other than improving students’ experiences and outcomes. Under the banner of the new Theme, which will run to 2020, we want to cut a distinctive Scottish path through this data landscape, ensuring that at the same time we consider other forms of evidence, such as qualitative data generated by students’ unions themselves. We want to think deeply about what constitutes appropriate and reliable evidence, and to ask questions that have not been asked before, such as whether there are unexplained variations in experiences and outcomes for students with different characteristics.

For the first time our institutional teams included colleagues from strategic planning. There is surprisingly little sector-level data for Scotland and the involvement of our planners will allow us to be more sophisticated as a sector in understanding our own data, thereby raising new questions. We were treated to a terrific presentation by Rona Smith of the University of Strathclyde, exploring the power and pitfalls of data and evidence. She reminded us that all measures are a proxy with complex influences. We came away convinced of the need to develop the capacity of staff at all levels, as well as our students and their representatives, so they can all gather, make sense of, and present (valid) conclusions from such data and evidence. Ultimately it has to be about using the best available evidence for the purpose of enhancing the learning experience and outcomes for our students. At the launch, Matt Adie of Stirling Students’ Union, showed how the written nominations from students for student-led teaching awards constitute a rich source of evidence for effective teaching and  learning.  

We also recognised the potential risk of a greater emphasis on data, particularly metrics, being used to promote competition and perhaps erode our ethos of collaboration. We know too that not all data constitutes evidence and that metrics are only the starting point not an end in themselves. We need to consider other kinds of evidence, especially that garnered from dialogue with our students, to achieve a full picture. At my own university, Dundee University Students’ Association, with support from Student Services, conducted its own survey of the issues that were most affecting students, presenting its findings to the University Court. This revealed that mental health issues were of overwhelming concern and has led us to agree to work together to address the issues highlighted.

An honest look

It also takes courage and trust for individual institutions, and the sector as a whole, to be honest about where improvements are needed, working collaboratively to co-design interventions and test their effectiveness. To take one example, graduate attributes formed the focus of several Themes and this led to all Scottish HEIs placing emphasis on the skills students acquire at university beyond their formal curriculum. Scottish graduates have higher rates of graduate employment and further study than the UK average, though there is no doubt further work to be done to ensure that data of this kind is robust evidence of the effectiveness of our Theme work. It is our intention that the new Theme will provide a lasting legacy of greater capacity to use evidence for enhancement, and at the same time generate better evidence of the effectiveness of the Themes approach.   

The careful stewarding of the sector by QAA Scotland over many years has created something precious and unique that we are determined to preserve and strengthen. I came away from the launch optimistic that the Scottish sector is absolutely committed to its collegiate and enhancement-led approach of deep partnership with students. This is how we will continue to improve the experiences and outcomes of all our students.