We’re sure, like us, you’ve seen it all in past weeks; from articles suggesting AI can create academic papers good enough for journals, to lecturers being urged to review their assessments in light of ChatGPT’s disruptive capabilities.
But are AI text generation tools really the problem? Or do they reveal more serious issues around assessment practices and the academic/student relationship?
If we continue with current assessment methods, there’s no clear solution on the horizon to mitigate against the use of AI tools. We’ve seen some efforts to employ the “detection tool” approach used for other forms of academic malpractices – but every single one of them has been beaten in practice, and many flag the work of humans as AI derived.
Simply restricting access is not an option – the technical landscape is moving quickly with Microsoft and others releasing a range of AI enhanced tools (such as Bing Search with ChatGPT) and platforms (such as Microsoft Teams). A myriad of new AI large language models (LLMs) are in the works, or soon to be released, such as Google’s Bard, or NVIDIA’s NeMo. Moving beyond text, LLMs are moving into sophisticated image and video generation. It’s impractical and improbable for students to avoid using such tools as they become ubiquitous in society – a “ban” on such tools would make about as much sense as banning the use of Word’s spell checker due to how ‘frictionless’ they are designed to be.
A starting point for many is that academic integrity is tarnished through students engaging in malpractice (however defined), and even if we accept this deficit framing, it presents a range of both practical and moral problems, as Jan McArthur has argued on Wonkhe in the past in relation to use of plagiarism detection software. Outside of academia, graduates write and generate ideas drawing on a range of sources using a wide variety of tools and approaches – we need to be considering supporting students in using AI as a part of effective academic work and preparation for graduation (and the participation in work and wider society that follows).
Protect, or accept and progress
“Computer-based maths”’ is a movement with the aim of reforming maths education. The principle is that curricula (and by extension, assessment) should “assume computers exist”, and move students towards a more developmentally effective learning experience. Higher education as a whole cannot get away from “assuming AI exists” and that students will use it – so there is a need to build curricula and assessments around this reality.
So rather than simply focusing on protecting the status quo, we could be working with students to design the assessment of the future. Academics have a responsibility to teach students how to use all kinds of tools and resources (search engines, academic literature databases, open data) in a useful but moral and ethical fashion – and already design assessments with a core focus on academic integrity, but augmented with a sense of trust and an understanding that these tools exist and will (and should) be used.
One of fundamental shifts in assessment is therefore likely to be around defining the level of creativity and originality lecturers expect from students, and what these terms will mean. The future of knowledge work is likely to expect people to range across multiple domains of expertise – so asking the right questions (whether this is of people, data, or artificial intelligence) becomes paramount.
For instance, we can see that the current iteration of AI tends to “hallucinate” many things, providing phrased answers that lack factual rigour. There’s been numerous reports of AI generating fake references or inventing entirely spurious facts. Although it is claimed that ChatGPT can write an essay deserving a 2:2, in practice the user requires a great deal of critical reasoning and research to bring the average response up to that standard. Such skills have always been in demand – and it is possible to imagine that “improving” an AI answer in this way as an interesting assessment design.
The QAA recently published a briefing note to support the challenges of academic integrity that the sector is facing. Its advice is clear: communicate early with students and discuss the technology with them; design assessments around co-creation, iteration, and critical thinking skills; and that detection tools are unverified and ineffective. This semester’s forthcoming assessment period may be turbulent across the sector; we’re never returning to pen and paper assessment, just like we’re never going to deny electricity exists. But surely a culture of academic integrity can only be built upon mutual trust and understanding; students don’t inherently wish to cheat, and framing these tools as cheating doesn’t help anyone.
The next steps
We clearly cannot design assessments based on student access to a paid product without planning to ensure all students can access it on the same terms. We can’t build learning and assessment around tools that could disappear or change radically at any moment. We need to assess students based on what they can do, not what they can afford.
As with any new technology there are risks and opportunities to the use of AI in education. Older readers may recall similar concerns as the internet became widely accessible – and downloading an essay from a website replaced the dominant model of buying one from a student in the year above. The technological arms race that followed brought about the rise of the essay mill, and some of the most effective academic malpractice that money could buy. Now that approach has been banned, maybe the latest and greatest AI will pick up the premium misconduct mantle – or something else will arise on the darker corners of the web.
A failure to iterate the design of assessment, and a reliance on the technology of “detection” as a stopgap doesn’t just leave academic integrity in question – it poses fundamental questions about structural inequality too.