OfS and Advance HE assess the state of the sector for AI in education

It’s hard to recall now, but among the cluster of policy micro-initiatives announced in the government’s post-16 education and skills white paper, was a pledge to “support the Office for Students to assess the impact of artificial intelligence, including how students are using it in assessments, to ensure the integrity of higher education assessments and qualifications are not compromised.”

OfS has today published the results of a research project conducted with Advance HE on the sector’s use of AI as it relates to students and education, and some brief additional regulatory guidance on how AI might show up in regulation of academic quality which, apparently, people said they wanted.

In conversation

The research report is based on a series of qualitative interactions, including online and in-person round tables, workshops, and “listening events”, plus a series of interviews, amounting to the views of more than 300 people, including 91 students or student representatives, from at least 127 institutions. It considers the opportunities and risks of AI in relation to teaching and study, assessment, AI literacy and skills, and strategy and operations.

We’ve been following this debate fairly closely ourselves over the last few years, and so the conclusions of the report read, to me at least, as being fairly obvious:

• The extent to which it’s a problem whether AI shows up in learning and teaching depends on whether it’s deployed to support or substitute for thinking
• AI poses a significant risk to assessment’s ability to demonstrate evidence of students’ acquisition of knowledge and skills – and relying on AI detection to either deter students from using it or to manage the issue post-hoc is a dead-end strategy
• Effective and responsible use of AI can only be achieved when staff and students have the requisite levels of AI literacy
• AI tools can most readily be employed at individual level to support personal productivity gains; wider operational augmentation requires strategic deployment

The picture painted across the four domains is a very familiar one – provision is uneven, there’s not perfect clarity about where AI is being used, and how, approaches to securing academic integrity vary enormously, lots of people have strong views about what is appropriate, or not – and where there is support on offer takeup is variable. Students are anxious about declaring their own use at the same time as they suspect their peers are enjoying illicit benefits.

All our own conversations support this picture – but there is also context to be aware of, such as the rapid pace of technological development and the proliferation of applications of different forms of AI, the financial challenges facing the sector, and the broader political and economic context for AI adoption. All of these factors make securing equitable access, pedagogically and operationally effective AI use, and assessment validity something of a moving target.

It’s bigger than us

The accompanying guidance note on the implications for quality assessment explains that institutions must take steps to ensure students develop appropriate AI literacy and subject and industry AI knowledge, that assessment must be designed in such a way as to validate students’ knowledge and skills (with the assumption that secure or closed assessment is likely to be the most effective way of achieving this), that there must be oversight of where AI is used in giving student feedback, and so on.

I suspect that many, on reading this, will be torn. It’s unarguable, and it’s absolutely the regulator’s job to point out, that institutions are responsible for securing quality and standards, and the arrival of a paradigm-changing new technology doesn’t let anyone off the hook. You’d certainly not want any provider to shrug its shoulders, stick to the usual approaches and assessment formats, and turn a blind eye to any rampant cognitive offloading going on. My personal view about the arrival of AI is that it means that institutions will need to double down on effective curriculum and assessment design and pedagogy capability among academic teaching staff – and those that weren’t particularly optimising their academic recruitment for those skills in the first place are now finding that the mountain they have to climb is somewhat higher.

But it’s also very obvious, in a way that isn’t especially foregrounded in this report, that institutional education leaders and their teams are sweating bullets to try and develop and deliver an AI strategy for education in their institutions that not only secures academic quality in the short term, but also leaves space for adaptation to the large-scale changes to teaching and learning that many suspect AI (in combination with numerous other social, economic and cultural forces) will bring. And, moreover, that most are doing so without a roadmap, with considerable internal ambivalence and/or resistance, and without much in the way of funds to invest in getting it right. The “how” is much more useful than the “what” in these circumstances – and the minimal examples of practice cited in the report illustrate the extent to which the answers are still in development.

There has been thoughtful research on how to achieve those expectations that has helped to drive the conversation forward in a positive way towards making accommodation with AI and making the most of the opportunities it offers, while managing the risks – and much of it is cited in the OfS/Advance HE report. As is traditional, more of that is needed. Advance HE has pledged to roll out a programme of support for the sector in managing AI in learning and teaching, focused on peer learning, and the development and sharing of effective strategies.

There is merit, especially where institutions are not already deeply enmeshed in that literature and the sector-wide conversation, in mapping out the issues and setting some expectations around good practice, even if it reads as both simultaneously baseline expectation and wildly aspirational. But there could be a bit more of an acknowledgement of the scale of the transformation that institutions are grappling with – and allowance for there being inevitable bumps on the road to quality nirvana.