Rob Summers explains the results of UK higher education’s first large-scale randomised controlled trials of learning analytics-prompted interventions
This article is more than 2 years old

Data analytics are changing the world as we know it, from influencing what we see on social media to improving public health.
Similarly, analytics have the power to highlight issues and prompt change in higher education and contribute towards reducing equality gaps between student groups.
In particular, learning analytics systems which use student-generated data – for example, data about attending lectures or checking out library books – can help to understand student engagement and learning.
When this data is combined with demographic data, analytics can help providers identify equality gaps; monitor the success of interventions designed to reduce equality gaps; or, for example, to understand the effects on engagement and attainment of sub-groups of students when pandemic-related lockdowns force face-to-face teaching and assessment to suddenly pivot online.
This student-generated data can be used to proactively identify students who may be at risk of withdrawing from their studies or failing some (or all) of their course due to short-term periods of low- or no engagement. When that happens, how can higher education providers help students reconnect with their learning?
Increasingly, higher education providers are contacting these students and directing them to support services either through email alerts or phone calls. But how effective are these interventions? That was the question that two TASO-funded randomised controlled trials, at Nottingham Trent University and Sheffield Hallam University, tried to answer.
After the intervention, student engagement – as measured by each institution’s learning analytics system – was pretty much identical regardless of whether a student only received an email or also received a phone call.
These trials are the first large-scale randomised controlled trials of learning analytics-prompted interventions in the UK – the Nottingham Trent University analysis included over 2,100 students – and we’re confident in the robustness of the findings.
Before you rush off to cancel your institution’s phone-calling service, there are a few things worth noting that complicate the interpretation of these results.
Post-intervention interviews with students indicated that the phone-call contact made them feel like they mattered to the institution. Support is one of four foundations of belonging at university and students taking part in the trials positively welcomed the personal nature of the coaching phone call because it felt like “the university cared” – though one student framed it rather as the “kick” they needed!
It’s also important to note that the duration of the trial over a single autumn term lacks the long-term outcomes of attainment, continuation, and progression.
And finally, because there was no comprehensive tracking of students’ access to support services, we don’t know if the signposting was effective or if the support services themselves were meeting student needs.
Perhaps the relationship between belonging and engagement rating is complicated, and the latter are not sensitive enough to the effects of a phone call in the short-term. Maybe long-term, the attainment, continuation and progression data will reveal some positive effects of the phone call on student success.
While the causal evidence base for learning analytics systems in supporting student success is weak, these platforms can be powerful tools for evaluating student-support interventions. However, current systems lack capabilities to facilitate evaluation such as randomisation of students to different support streams or to different kinds of intervention, and subsequent reporting on student outcomes other than attainment, such as use of support services.
Nottingham Trent University vice chancellor Edward Peck and the Department for Education student mental health taskforce have identified that analytics can play a role in identifying students in need of mental health or wellbeing support. It will be crucial to robustly test the impact of any wellbeing interventions that analytics systems may trigger.
TASO will continue to explore the potential for better evaluation post-entry support including how to use and develop institutional data infrastructure to facilitate robust causal evaluation of post-entry student success initiatives.
Afterall, analytics have the ability to change the world. But analytics alone aren’t enough.
When used effectively and consistently, analytics can highlight where there are issues and where interventions can be applied. And, paired with action and evaluation, analytics can galvanise the changes we want to see – including closing equality gaps in higher education.

Jim Dickinson | Analysis | 4/09/26
Ola Thomson | Analysis | 27/08/26

Wendy Ashall | Analysis | 26/08/26

Jim Dickinson | Analysis | 25/08/26

David Kernohan | Analysis | 12/08/26

Beth Hocking | Analysis | 23/07/26