Mindful of its statutory role in Northern Ireland, Wales, and Scotland – and the vast amount of quality assurance work it performs overseas – the Quality Assurance Agency (QAA) decided in July 2022 to demit itself from the Designated Quality Body (DQB) role in England.
The reasoning was straightforward – the Office for Students (OfS) would not permit the QAA to perform the DQB role in a way that was consistent with international standards, thus putting in jeopardy the QAA’s place on the European Association for Quality Assurance (ENQA) register that is a requirement for so much other work.
And that, however we feel about it, should have been that.
However, the Office for Students can never resist getting the last word – and today’s release of a “summary” of the triennial report on the Designated Quality Body is both petty and unnecessary.
If you are reading this article as a member of the Commons Education Committee, or indeed at the Department for Education, you really need to consider whether this is the action of a responsible regulator.
Playing by the rules
So what is a “triennial report”? It’s all there in section 9 of Schedule 2 of the Higher Education and Research Act – a report (developed by the Office for Students in consultation with registered providers, student representatives, employers, and others) on the work of the DQB sent in confidence – to the Secretary of State.
The Office for Students shared a provisional report with the QAA in April 2022 – there’s no indication in the summary or in the board papers that the views of others were sought, as is required in HERA. The report was finally submitted to the Secretary of State in November 2022, some five months after QAA’s decision to demit (and a little more than a year after it was due). If you are thinking that’s because of a need for scrutiny by OfS’ Quality Assessment Committee – note that the QAC met on 12 September.
Clearly the Secretary of State isn’t going to publish the report now, and there is no statutory or procedural reason to publish a summary of the report – though OfS claims that there is:
a strong public interest in relation to our judgements about the performance of the QAA in its role as the DQB
Other than a brazen attempt to sway popular opinion ahead of the required DfE consultation on the immediate future of the DQB role, it is difficult to find any evidence for this.
Delegated authority
Plunging back through the OfS board paper archives it is very difficult to see evidence that QAC (or indeed the Board) ever got to discuss the DQB triennial review before it reached a near-final state in late March 2022. It turns up as being reported to QAC on a few occasions before that, with the majority of content under that heading redacted (“public interest”, remember?).
In the March minutes we get a heavily redacted write-up of the discussion that suggests:
the board’s comments would be taken into account in revising the draft report.
And we also get a curious agreement that all further activity – which we know must have included significant changes based on both Board comments and the response to the draft version from the QAA – on the triennial report would be conducted under delegated authority. Why wouldn’t the board want to see the revised report?
Happily, the October 2019 designation agreement is available – and we can see that QAA was charged with developing guidance for the various DQB functions and presenting this to OfS by 6 September 2019. Clearly this either happened as planned, or was delayed by mutual agreement because the Board minutes of 26 September 2019 note:
The chair of [QAC] confirmed that, following previous difficulties, the OfS’s relationship with the Designated Quality Body (DQB) was good.
This, of course, was linked to work undertaken to support old regulatory requirements. OfS decided for whatever reason to tear these up in 2021-22 via a consultation, so we can only conclude that the “three successive submissions” the OfS received from QAA occurred over the five months between the consultation response being confirmed in March 2022 (surely OfS would not pre-empt the conclusion of a consultation?) and the QAA demitting from its DQB role in July 2022.
More germanely, we can be sure that this was not the opinion of the board or QAC, who lost control of the process after the March 2022 board meeting and apparently did not have the opportunity to feed in on this issue.
Law of the playground
Even if the QAA wanted to still be DQB, OfS wouldn’t let it, because – well, here are the reasons.
1. The quality of the DQB’s work
The assertion here was that the reports that the DQB produced were not of the required quality for use in regulation. However, OfS did in fact use these reports (on both quality and standards) as a basis for regulatory decisions – on degree awarding powers, university title, and entry to the register (as a recent example, OfS used a specially commissioned report regarding standards at Multiverse in 2022, leading to entry to the OfS register). So if you’ve been turned down by OfS on any of those counts, either you just got some rock-solid grounds for appeal or the regulator did indeed have enough information from the reports to confidently make a decision.
Getting stuck into the reports themselves (for example here’s the Quality and Standards Review for the National Centre for Circus Arts from 2022, and here’s that bespoke Multiverse Standards Review for the same year) you are struggling to see any evidence to back up the OfS assertion that:
We have been unable to make registration decisions with confidence, for example, where a QAA assessment report contains some negative judgements but does not articulate the reasons for these sufficiently clearly.
You can see clear and detailed reasoning for specific judgements in each report – indeed the DQB has set these out for QSRs in a table at the top. Of course, it is OfS that gets to decide whether these reports are published openly – there may be others we are not seeing but it feels unlikely that they would be different in form or approach from those we can see.
It’s also notable that OfS claims that:
Our view is that the QAA’s performance has not been acceptable during the most recent year (1 April 2021 – 31 March 2022).
Had the OfS stuck to the initial timeline for the triennial review, this year would not have been in scope.
2. The DQB’s methods
These, to be clear, are not the methods under which the DQB conducted assessments for the OfS for the majority of its lifetime. These are the revised methods linked to changes to the regulatory framework that only went live after the QAA demitted from the role.
Incidentally, these methods were actually approved by the OfS – to the extent that the regulator said:
We look forward to continuing to work with the QAA until March 2023 as it undertakes assessments of quality and standards to inform our regulatory decisions (my emphasis).
Neglecting, somehow, to mention that it was unhappy with the methods the QAA would use and the quality of the reports it would provide.
The criticism is re-expressed elsewhere in the documentation. For example – in the lengthy justification for why OfS has been cheerfully using assessments it now claims are not suitable for regulation:
For example, we have used information contained in an assessment report that contains positive judgements despite that report not being consistently evidenced or focused on outcomes and where there is no other evidence of increased regulatory risk.
The curious line here is about a report not being “focused on outcomes”. In the lengthy and – for OfS – highly embarrassing court battle as regards Bloomsbury Institute the regulator is very keen to make the point that the DQB is not at all (and can never be) involved in the assessment of (B3) outcomes:
The OfS did not seek evidence from the QAA in relation to Condition B3, and said in its registration decision that it was appropriate to place little, if any, weight on the representations regarding the QAA annual monitoring review. The OfS also confirmed, in these proceedings, that the OfS did not have regard to the detailed scrutiny by the QAA as part of the degree-awarding process.
The justification here was in paragraphs 349- 350 of the regulatory framework:
The OfS will consider the assessment made by the DQB [the Designated Quality Body for assessing the standards of providers under HERA, s23, namely the QAA] when determining whether initial conditions B1, B2, B4 and B5 are met. Where the provider has a track record of delivering higher education, the OfS itself will assess whether the provider is able to satisfy Condition B3.
It’s not clear on what basis OfS was expecting DQB reviews to be outcome-focused given that it has repeatedly made it clear there is no role for the DQB in B3.
3. Conflicts of interest
QAA is a membership organisation, and boasts the majority of UK universities as members – indeed membership has significantly expanded since 2019. The OfS believes this to constitute a conflict of interest – which is odd as the esteem that the QAA is held in by the sector (as evidenced since by this high level of voluntary subscription) was a key facet in its selection as DQB.
We’re not given any evidence as to where and how this conflict may arise – especially given that the DQB arm of QAA is run as a separate entity with an independent board. There’s a suggestion in the text of the summary, that the QAA:
publishes guidance and reports in its role as a voluntary membership organisation and makes public statements that advocate for policy positions that are not, or may not be, consistent with the OfS’s approach. Providers that follow this guidance may incorrectly assume that they will satisfy the OfS’s regulatory requirements as set out in the conditions of registration.
The QAA has been a membership organisation, and has supported sector owned activity like the Quality Code, the Subject Benchmarks, and liaison with professional bodies, throughout its lifetime. It is notable that the designation agreement does not attempt to restrict, or shape, any of these activities. It is not clear how the QAA doing things in 2018 was not even worthy of note, but the QAA doing the same things in 2022 places it at a risk of a conflict of interest.
4. Value for money
Yes, OfS appears to want to go there. In 2022-23, your basic mandatory DQB subscription for a provider with 20,001 to 30,000 students (band M) was £12,413 (not including a rebate from the previous year). In contrast the mandatory subscription to the Office for Students would be £181,000. If we are to believe the rumours of a 13 per cent rise in these OfS fees (justified at least partially by OfS taking on DQB functions) there’ll be an extra £23,530 this year. Value for money?
The QAA is OK
More generally, the QAA engenders a great deal of UK and international respect for the quality of its work. Indeed, in 2017, OfS recommended the QAA as the only suitable candidate for the DQB role – and it received “overwhelming support” from the sector. Despite there being no requirement for higher education providers in England to be fee-paying members of the QAA (above and beyond the statutory DQB functions), the overwhelming majority of large providers are members. When the sector faced a real quality and standards crisis during the pandemic – where it was impossible for providers to deliver professional qualifications – it was QAA that addressed and solved the problem. The QAA, it is safe to say, is quite good at quality and standards assurance.
An unpreceded joint statement from DfE(NI), HEFCW, and SFC adds:
The Scottish Funding Council, the Department for the Economy in Northern Ireland and the HEFCW have worked in close partnership with QAA for many years in creating, maintaining, and strengthening quality arrangements that are bespoke to the Northern Irish, Scottish and Welsh higher education systems. Over that time, QAA has consistently delivered high quality work that meets the needs of the nations, operating in collaboration with the sector and applying its expertise and experience. We have confidence in QAA as a valued and internationally respected quality assurance agency and look forward to continuing to work with QAA in the future
It is not clear why the Office for Students has chosen to publish a summary of a document that has been superseded by events, or what “public interest” is served by this decision. You do have to ponder the likely authorship of the lines in the post-legislative memo that say:
Assessments of standards must be made against sector-recognised standards. While this ensures the sector is adequately involved in setting standards, the OfS finds that this may constrain their ability to take a more direct regulatory approach
It’s far from clear that an environment in which the sector is moved away from sector-recognised standards to a “more direct regulatory approach” would be a healthy one, or one that pays due regard to the autonomy of higher education providers. A forthcoming statutory DfE consultation on the de-designation of the designated quality body may offer the sector chance to feedback on ideas like this.
But the one voice we won’t hear is that of the DQB itself, the QAA. As the statement reads:
QAA has contested the content and conclusion of OfS’s triennial report, and made representations against it. In QAA’s opinion the final report, and the published summary, substantially misrepresent QAA’s record as DQB. But we do not think it serves the interests of students or the sector for QAA to engage in a protracted public disagreement with OfS, and so we do not intend to publish a detailed rebuttal or to make further statements on this matter at this time.
I don’t know or care what’s behind Susan Lapworth’s vendetta against the QAA, but it must be exhausting for her. Someone should tell her no one believes the stuff she makes up about them anyway.
The dignified response and conduct of QAA is a stark contrast to the OfS approach, the latter of which abdicated its responsibility for failing to define what it wanted from quality assessment because it simply doesn’t know how to achieve this : it does not take account of stakeholder views; student engagement is tokenistic; there is no visible engagement with other external QA agencies in terms of learning from or even contributing to a wider debate; it does not represent value for money under ANY measure. It has been a sourly orchestrated, missed opportunity to build something positive for the international reputation of HE and be appropriately critical of those it regulates. Time to close the drawbridge please on Northavon House before more long lasting damage is done.
The OfS performance and behaviour is increasingly out of control.
Their weirdly defensive blog on building better relationships with the sector – which basically is “sorry if you’re upset because we’re better than you” today is another example. Someone needs to get a handle on them and start them from scratch…
Feels like they’re not far away from employing the “look what you made us do” line at this point.
It’s all very, ‘No, I dumped YOU first!” Whilst the QAA appears to have taken the rather more dignified high road in public fora. Combined with Ms Lapworth’s rather passive-aggressive statement today about the new era of regulation and how the OfS absolutely had to release so many consultations due to the level of risk, it does make you question the sanity of England’s regulatory approach (if you did not already).
I do feel rather sorry for many of the OfS staff. It can’t be easy working in an environment where the strategic direction makes no sense (largely because it follows government policy).
Interesting that OfS have the capacity to do this rather unnecessary piece while the rather necessary and long overdue review of the DDB remains just that.
Very strong jilted lover vibes… My God how pathetic.
I am surprised the OfS thought this would go down well just at the moment they say they are trying to reset their relationship with institutions. Doesn’t bode well for the reset.
Time for a reboot. With an emphasis
On the boot
Says it all when my partner and I read the rather petty comments from the OfS which included, on the BBC website a statement about it being ‘independent’. Neither of us realised it’s supposed to be ‘independent’. We both assumed it to be the mouthpiece of whichever minister is in charge at the time – as evidenced by the way some VCs bounce around from initiative to initiative which they inflict on their poor colleagues.
Lapworth: arrogant, over-promoted and incompetent. Treatment of staff in the set up phase of OfS was a clear indicator of the lack of judgement, inexperience and ineptitude to come.