Set aside for a second the fact that when you ask students if academics should be able to use AI in marking, they tend to be vehemently opposed.
I’d set it aside partly because asking questions like that in surveys ignores all the subtleties. What sort of marking? Will humans be involved at any stage?
What if it improves consistency and predictability? What are the trade-offs between spending money on humans doing that versus other things?
They need a conversation, not boxes to tick.
From what I can tell, many of those conversations in working groups are dominated by simplistic binaries and camps, what counts as “student opinion”, and the usual tensions between university-wide policies and everyone else’s claims to autonomy.
What’s missing from much of the discussion is the sacrosanct get out of jail free card of academic judgement.
And assuming universities aren’t keen to inadvertently trade away their right to deny an appeal on that basis, it needs to be front and centre.
Even where staff use of AI does come up, it tends to be framed as a transparency problem or a mixed message.
In our recent work with Studiosity on academic integrity, “staff using AI to mark and provide feedback was cited as a particular issue” by institutional leaders – but as a question of what students are being told about AI, with nothing on what the university might be giving up.
Under the Higher Education Act 2004, the Office of the Independent Adjudicator (OIA) can’t consider a complaint “to the extent that it relates to matters of academic judgment”. The courts have long said much the same – any view of theirs on what mark or class a student deserves would be “jejune and inappropriate”.
But the protection attaches to expertise. The OIA defines academic judgement as “a judgment that is made about a matter where the opinion of an academic expert is essential”, and is clear that a judgement doesn’t qualify just because an academic made it. Its draft new Scheme goes further, describing it as a decision only someone with the appropriate expertise can make.
The word doing the work is “someone”. The whole protection rests on an expert having looked at the work and formed a view, which is why a student can’t argue with the view – and why they can argue about whether a view was ever formed.
The facts of a case, and the fairness of the process that produced a mark, aren’t academic judgement, so the OIA can look at them. And “who, or what, decided this mark?” is a question of fact.
One of the OIA’s AI case summaries from July last year shows how this might play out. An MBA student argued that AI had written their dissertation feedback. The provider said it hadn’t, and the OIA found it reasonable to give more weight to the marker’s account of how they’d marked the work than to a detection tool.
Fair enough – but once AI is formally part of the marking workflow, the marker’s account changes, and so, presumably, does the defence.
The usual answer in working groups is a human in the loop. The model does a first pass, the academic checks it and signs it off, and judgement stays, as one vice-provost put it on this site in the summer, “firmly with the educator”.
Anyone who’s ever been handed a draft knows what happens next. Once there’s a number in front of you it becomes the default, and disagreeing with it becomes work.
Ofqual’s January working paper on AI in marking backs that up. It describes the risk of humans reviewing AI marks as an “anchoring effect and/or risk of uncritical human review”, warns that “human performance at checking may not equal their performance on the original task”, and worries about process designs that let the human “omit the thinking required by the original task”. Marking in parallel, it says, “avoids anchoring”.
A randomised experiment published in PNAS Nexus in June gave 1,339 teachers in Greece a piece of student work with a deliberately wrong mark of five out of 10, labelled either as a colleague’s or as an AI’s.
Where the mark was too harsh, teachers were less likely to correct it when it came from the AI – the gap from the right answer was 22 per cent larger. Where it was too generous, they corrected both just as readily.
So the deference runs in the direction that hurts students.
There’s also a line in the Ofqual paper that weighs up the efficiency case – if a human properly reviews every response, there’s “no efficiency gain”.
The savings from a first pass only materialise if the academic doesn’t really redo the thinking, which is exactly when it stops being their judgement.
A delegate at our Senate Summit last month made the same point from the other end – if AI does the marking and someone still has to check it, the cost has moved somewhere less visible.
The more sophisticated pitch is about consistency. Human markers are notoriously variable, the argument goes, and a model applying a rubric would at least be predictable.
It isn’t. Even on the setting meant to make them repeatable, language models can and do give different answers to the same input.
Researchers at Thinking Machines ran the same prompt 1,000 times on that setting, got 80 different answers, and traced the cause to the fact that “the load (and thus batch-size) nondeterministically varies” – so the answer can depend on how busy the server is. Ofqual adds that performance shifts when the underlying model gets updated.
We do accept inconsistency in humans, but we accept it as disagreement between experts. Ofqual’s rule for school exams has been that “where there is a legitimate difference of opinion in application of the mark scheme, the original mark should stand”. There’s no equivalent for the same marker disagreeing with itself about the same script on a Tuesday afternoon, for reasons nobody can explain.
Unlike human variation, it can also be proved. Double-marking is getting rarer, but anyone can run the same essay through the same model 10 times and screenshot the spread. If arbitrariness is what takes a decision outside academic judgement, that’s an appeal that writes itself.
A human marker can be asked at appeal to explain a mark and stand behind it. A mark generated in January on a model that’s been retired by June can’t even be regenerated.
Services still have to be performed with reasonable care and skill. Courts won’t rule on a grade itself, but the High Court in Siddiqui said negligent teaching methods are actionable in principle, and that “simple operational negligence in the making of educational provision” is fair game.
The test is roughly whether a responsible body of practitioners would do what the university did – and with some now allowing AI-assisted grading subject to review, a university could argue that such a body exists.
But the practice also has to stand up to logical scrutiny. That gets harder when the schools qualifications regulator says that using AI as “the sole mechanism for awarding marks does not comply with our current regulations”, and that sign-off models invite anchoring.
Since February, the UK GDPR has said that a decision is solely automated “if there is no meaningful human involvement in the taking of the decision”. Universities can make significant automated decisions about students, but only with safeguards – including a right to “obtain human intervention” and to “contest such decisions”.
So a rubber-stamp sign-off would put a university inside the regime it was presumably trying to stay out of. And it’s hard to see how an appeals regulation could wave away a statutory right to contest a decision just by saying the words “academic judgement”.
Marking multiple choice, checking whether references exist, and spotting arithmetic errors on a mark sheet are mostly not academic judgement in the first place – they were always reviewable, and mistakes there get caught.
The evaluative calls – the kind Ofqual says are hardest to mark – are where the shield is strongest. Putting a model there puts the least reproducible tool where the law allows the least scrutiny.
If a university does want AI in the evaluative bit, one version flips the order. The academic marks first, blind. The model marks in parallel, and big gaps go back for a second human look – in effect, AI as a moderator.
Students are told, and can ask for human-only marking, as they can at Newcastle in Australia. Every mark logs the model, version, and prompt. And AI can tidy an academic’s comments – the King’s, LSE, and Southampton line – but can’t originate the judgement behind them.
The bigger ask is that working groups put the question on the table before they get anywhere near tools.
For every proposed use, could the university still say with a straight face that the mark is a matter of academic judgement – and would it be prepared to say so in front of the OIA, or a judge?
Bottom line? Students have been told that work produced by a machine isn’t really their own.
As such, universities shouldn’t be surprised when a student turns up with the same argument about their mark – and unlike a marker insisting that an essay smells like AI, the student might actually be able to prove it.