Earlier this month over 450 delegates from 26 countries joined QAA Scotland’s International Enhancement Conference, “Building Resilient Learning Communities: Using Evidence to Support Student Success”.
Contributors to the event included colleagues from South Africa and New Zealand, both places who have implemented their own Enhancement Themes. Emulation is a sincere form of recognition indeed. If popularity alone were evidence of impact, the Scottish enhancement-led approach would surely win awards. The sector has staunchly supported its Enhancement Themes and wider the Quality Enhancement Framework (QEF) which is often characterised as collaborative, collegiate and distinctive.
What are Enhancement Themes?
Back in 2003-4 when the QEF was first implemented, the Framework itself and the Enhancement Themes in particular were described as “radical”. Although elements of the approach have been adopted elsewhere, Scotland remains the only country to have a coherent framework for both reviewing and enhancing the quality of the student learning experience.
So the Enhancement Themes have broad appeal and longevity. But do we know the approach is working? Is it really improving the student learning experience?
A desire to address these questions led the Scottish sector to launch, “Evidence for Enhancement: Improving the Student Experience” the Theme which began in 2017 and is drawing to a conclusion now. Evidence for Enhancement was one of the most ambitious and, some would argue, successful Themes to date. It began with a set of simple questions:
- What information do we need to help us identify what we do well and what we could do better?
- How do we know we are using the information we have in the best way possible?
- Do we have the right information?
What did we learn?
Over the three years the Theme has covered vast ground, from activities aimed at supporting staff and students to understand and use a range of qualitative and quantitative data – for example the enormously popular staff and students “Guide to Using Evidence” and the “Optimising the Use of Evidence” webinar series – to activities engaged in generating novel forms of evidence, articulating and evaluating what really matters about those elements of the educational experience that have no easy metrics, such as responding to the student voice, valuing the role of programme leaders and considering mental health and wellbeing.
In support of evaluating the difficult to measure, the Theme engaged a group of colleagues who produced a model for doing just that, Beyond the Metrics: Identifying, evidencing and enhancing the less tangible assets of higher education. The model has been tried in a number of Scottish institutions and in England at least one provider used it to evaluate their submission for the TEF.
In addition to the cornucopia of useful resources, the Theme engaged a wealth of enthusiastic staff and students. A student-led project resulted in a set of Principles for Responding to the Student Voice which have been turned into a pack of activity cards – now also available in digital and fully editable format – which can be used in a variety of everyday, as well as bespoke, settings. A group of careers professionals worked together to support graduate employability including developing a MOOC to help final year students and recent graduates present the skills they had learned while studying to future employers.
A partnership with a group of University Planners produced a ‘deep dive’ analysis of the Scottish sector’s outcomes from the National Student Survey. This work began from a desire to understand what underpinned the NSS results and, from there, to shape activity to enhance the student experience across the sector. Among the outcomes of the work was an identification of those survey questions which had the greatest, and which the least, impact on overall student satisfaction. The results were considered at a series of meetings involving the vice-principals and during a sector event focusing on student surveys. While we can’t be certain of the correlation – and this was not the motivation for carrying out the analysis – a strong performance by the Scottish sector overall in two key areas, the NSS results and graduate outcomes, has been observed in 2020.
Evaluating evaluation
To help the sector track the impact of the Theme, we engaged an external evaluator from the outset. Professor Liz Thomas is due to report her findings by the end of 2020. Helpfully, at the launch event, the sector engaged in a workshop activity to identify what success of the Theme would look like – this included some modest aspirations like improving staff and student confidence in understanding and using data, but it also outlined more ambitious plans including the Scottish sector being seen as a leader in its use of evidence to enhance the student experience. Recent articles on value emphasise the extent to which this remains in the eye of the valuer, but our work has been recognised for example the Guide to Using Data is cited on the OfS website and, although it makes us blush a little, it has also been hailed by some bloke called Jim Dickinson as the greatest development in student engagement in his career!
The sector has worked hard to support an earnest evaluation of its work – gathering a suite of light touch and in-depth case studies produced by each institution to accompany similar case studies for a variety of the sector-level projects. The aim being that we should be able to track intentions against outcomes, enabling us to produce evidence of enhancement that is rooted in institutional practice and, the holy grail, visible in the experience of students.
What is next?
The November conference was held at a point of transition between one three-year Enhancement Theme and another. The new Theme, Resilient Learning Communities, will run to 2023 at which point the Scottish university sector will celebrate 20 years of its enhancement-led approach. With this in mind, QAA Scotland is commissioning an external evaluation of all Themes offered since 2003. The total value of the contract is up to £100,000. The call for expressions of interest to evaluate the Enhancement Themes remains open to 30 November.
