The sector still needs a say on quality and standards

The term “quality wars” was coined by one of Wonkhe’s favourite writers about the business of higher education regulation: David Watson.

In a seminal 2006 article he characterised an ongoing process in which:

audit society and the accountability culture have collided (apparently) with academic freedom and institutional autonomy.

It is ancient history that I have covered on Wonkhe in more depth in the past, but suffice it to say that there are two positions about the way we understand (and measure) academic quality and standards:

• Quality and standards are a matter for academics and universities – who choose to collaborate in order to drive up standards within the sector
• Quality and standards are a matter for regulators and government – who enforce minimums with legislation and regulatory requirements.

Put like that, it strikes at the core of issues of autonomy among independent universities, and accountability to the government that was and remains the primary funder of higher education.

And a final distinction:

• Quality refers to how well providers support students to consistently achieve positive outcomes from learning, personal development and career advancement, while meeting the reasonable expectations of those students, employers, government and society in general.
• Standards are set and maintained by degree-awarding bodies the award of academic credit or qualifications. Providers with degree-awarding powers are responsible for defining their own standards (via pass marks, grading criteria, classification rules) to recognise student achievement. These should align to the expectation set by the sector itself, and by professional bodies

An uneasy truce broken

The role of the Designated Quality Body (DQB), as set out in the 2017 Higher Education and Research Act, was the latest in a stream of attempts to square this stubbornly round circle. The Act provided that the DQB would be formally endorsed by both the sector and the regulator – it would have sole responsibility for academic standards, and a key role in academic quality: including entry to the sector, the conferment of degree awarding powers, and investigation where concerns about quality and standards had been ranged.

The Office for Students always had responsibility for regulating based on the outcomes of these investigations, but over the six years it has been active it has gradually taken on more of the DQB work. This has happened both in plain sight (the regulatory conduct and treatment that resulted in the Quality Assurance Agency demitting from the DQB role, as documented in OfS’ dreadful triennial review), and more subtly, via the gradual co-option of “sector owned standards” into a document owned and updated by the regulator.

The argument was always one of quality and accountability, but it is notable that although OfS has carried out its own investigations it is yet to regulate on any of the findings (though,to be fair, it has found the ability to request action plans based on data-driven investigations). It has sought to own the interactions that used to be carried out by the QAA, and with the consent of the sector, but it has not put any substantial thought into how it would act faced with evidence that a large provider is letting students down.

It is fair to argue that the current approach to quality and standards hasn’t worked. The previous government seemed to get a press release about “low quality courses” out of the door every week, usually bolstered by a reminder of the fearsome power of a fully operational regulator (fines of “up to £2m”, restrictions on recruitment, specific conditions of registrations). Despite this arsenal of firepower, we have seen almost no direct regulatory action aside from recent and very narrowly drawn requirements that certain providers work to improve their outcomes statistics. By 2028.

Enter the Behan review

Fit for the future starts from the understanding that the Office for Students currently has responsibility for both quality and standards, and this role was formally supported by the QAA as DQB. Behan also notes that the QAA demitted itself from the role, citing the suspension of European Quality Assurance Register (EQAR) membership because of a failure by the English quality assurance to meet international standards – and notes QAAs continued role as a membership organisation in England and a formal component of quality assurance processes elsewhere in the UK.

In calling for “coherence” between the numerous strands of Office for Students quality assurance activity, the report makes the leap to giving the independent regulator prime responsibility for “setting and assessing quality and standards” in order to hold the sector to account.

It should not rely on a separate designated quality body.

To that end Behan recommends that the passages relating to the role of the DQB should be removed from HERA – the first and only time he directly recommends legislative change in the report. His rationale is primarily one of custom:

There is a strong precedent in many sectors for regulators to set their quality standards and regulate against them. Conversely, having a separate body carry out this function is highly unusual

The report does note sector concerns about equivalence with international standards, and to this end he recommends

OfS Reflects on how its standards and processes can demonstrate equivalence with EQAR requirements to enable English higher education providers to align to international standards

Back to the future?

Broadly the remaining recommendations concern the bringing together of three strands of quality-focused work – the annual examination of B3 data (baseline), the teaching excellence framework or TEF (improvement, more normally described as “quality enhancement”), and the deeper investigative work at an individual provider or subject (responsive, indicator or intelligence driven).

I often cop for good-natured abuse from my colleagues for my habit of harking back to the state of quality assurance two decades ago. Back then a cyclical system of subject-based reviews (assurance), the activities supported by the teaching quality enhancement fund (enhancement), and the power reserved by the former funding council to investigate or censure (based on the formidable “conditions of funding” which included the independent inspection regime, and including the ability to put a direct block on planned expansion).

Behan appears to be arguing for a similarly articulated system, but he puts the responsibility for all of it on an organisation that he describes in some depth as appearing insufficiently independent from government (though, to be clear Behan is satisfied that OfS is independent), and that does not have a particularly impressive track record in carrying out any of these responsibilities (to be scrupulously fair, some of the investigative reports aren’t actually terrible).

Other recommendations include the reframing of investigative activity as “routine assessment”, and making it clearer how investigations are run, what criteria institutions are being assessed against, and what criteria led to the selection of a provider for investigation.

The proposals also remain divorced from three other established systems of quality assurance that currently exist:

• The internal quality assurance mechanisms of higher education providers
• The system of external examiners
• The work of professional, statutory, and regulatory bodies in approving and “kitemarking” specific courses and providers for entry into professions.

Alignment with Europe

In arguing for ENQA membership, Behan implicitly suggests that the quality assurance regime in England will need to conform to the standards and guidelines for quality assurance in the European higher education area (ESG). However, there are a number of facets of ESG that the current Office for Student managed system is arguably in breach of:

• Institutions should undergo external quality assurance on a cyclical basis (1.10)
• Quality assurance should address the effectiveness of internal quality assurance processes within providers (2.1)
• Quality assurance should be carried out by groups of external experts, including… students, and employers or professional practitioners (2.4)
• If the agency takes any formal decision based on the reports, the decision should be published together with the report (2.6)
• Quality assurance agencies should be independent and act autonomously (3.3)
• Quality assurance agencies themselves need to undergo a cyclical external review against the ESG (3.7)

The review asks OfS to reflect on how it can demonstrate equivalence with EQAR. Some of the stumbling blocks are addressed by other recommendations on transparency and independence from government, or the place of students at the meaningful heart of regulation. But for whatever reason, OfS has set its face against the idea of cyclical review (though it has argued in the past that TEF is a cyclical review, TEF does not interface with or examine internal systems of quality assurance) – and recommendations double down on a data and intelligence driven risk based system that would not, and will never be, ENQA compliant.

Always work to do

There are two bodies that currently exist in England with a direct responsibility to work on teaching quality: the QAA on assurance and AdvanceHE on enhancement. Both were, in living memory, funded directly by the English higher education regulator to carry out these roles. Capacity currently exists within the ENQA-registered QAA (see, for instance the work on transnational education and on international foundation years, and the responsibility for quality assurance in devolved nations) to carry out this role again. The story on enhancement is a sadder one, but substantial expertise (and a number of sector networks) still exist within AdvanceHE.

In the schools system and the FE system the distinction between quality (Ofsted/ESFA) and standards (Ofqual) is distinctly drawn, and led by arms length agencies. In higher education a single government-backed organisation is asked, uniquely within the education system, to deal with quality, standards, regulation, financial sustainability, and student welfare. Reviews of arms-length bodies are not asked to examine existential questions – but there would be value in asking whether, if OfS did not exist, we would choose to construct it again in its current form.

But to return to my opening theme, a re-erupting quality war helps nobody. Every registered provider of higher education should be competent in the way it adheres to academic standards set by the sector. Every registered provider should be able to run an appropriate internal quality assurance regime that ensures students are getting a good deal. Most do. And it is much less burdensome to inspect, at the centre and on a regular basis, the functioning of these systems than to try – and fail – to ensure the quality and standards on every course of higher education in England from an office in the north of Bristol.