HEFCE press the quality reset button

HEFCE’s second phase consultation on ‘Future approaches to quality assessment in England, Wales and Northern Ireland’ has finally been published. The proposals have been widely trailed over the past few weeks, after the publication was delayed and copies of the proposals were leaked and discussed widely.

The story of the way in which these proposal have been developed is, I think, as interesting as the proposals that have now been published.

Back in October 2014, HEFCE announced that its contract with QAA was nearing an end and that this was going to be used as an opportunity to review the entire ‘Quality Assessment’ system. The new system would be “risk-based, proportionate, affordable, and low burden”. At the time, this was barely reported, other than on this site and I made a few supplementary comments on my blog as well. When it happened I found this approach surprising, but I wasn’t sure it was a game-changer. Now it is clear that it is.

It is still not clear why HEFCE decided to go it alone with these proposals and not in partnership with UUK, GuildHE and NUS which is the usual and well established practice for the development of quality assurance arrangements to date. It’s notable that Wales and Northern Ireland are tacking relatively close to England, but carving an increasingly independent path. It is equally notable that Scotland is not joining in at all. I can only conclude that if these proposals go ahead, there will no longer be a UK-wide system of quality assurance in higher education. That prospect, I think, requires more attention, and urgently.

So now that HEFCE has its cards clearly on the table, a quite substantial restructuring of the external quality assurance environment is being proposed. There is a lot here to digest, but by my reading the proposals boil down to the following five areas:

1. Ending the cyclical process of external audit/review for universities and other HE providers
2. Deconstructing the UK Quality Code
3. Shifting the focus of external QA from process to outcome
4. ‘Strengthening’ the External Examiner System
5. Creating a gateway for new providers of higher education to enter the sector

1. Ending the cyclical process of audit/review for universities and other HE providers

In broad terms, I have some sympathy with HEFCE’s starting point here. I have worked at two universities that have been through a number of cycles of institutional audit and review, as well as other forms of engagement such as reviews of trans-national education (TNE) or collaborative provision. Many universities have also gone through the Degree Awarding Powers process during this time. I think it’s fair to say, and I don’t even think QAA would argue this point, that those reviews have not evolved a great deal and while still of some value, they have reached a point of providing diminishing returns where established universities, their students and the rest of the sector are learning less and less from participation.

The proposals in HEFCE’s consultation suggest that the review cycle will be ended with responsibility for QA being more explicitly held by university governing bodies. It is stipulated that this responsibility will be exercised through an annual review and report on key student data sets that will in turn be returned to HEFCE (this is quite clearly modelled on the financial and value for money reporting structures that are currently in place). But crucially it is proposed that HEFCE will roll QA into its own, existing, five-yearly cyclical assurance reviews to check the evidence and processes used to produce the annual statements.

So, while there is definitely a debate to be had about whether cyclical reviews still have value and if they do, then how they should be conducted; I don’t think it can be concluded that these proposals reflect a serious and deep engagement with those issues. The burden will not be lifted, it will merely shift from external review by QAA to external review by HEFCE (with added burden of annual submissions).

2. Deconstructing the UK Quality Code

The UK Quality Code is not explicitly referenced in the proposals, other than a passing mention of the Framework for HE Qualifications.

Similarly to QAA external review, I think the Quality Code is now, after 15 years or so, producing diminishing returns, as its contents merely reflect existing practice that is well understood and deeply embedded in institutional structures. It does however provide a useful purpose as a reference point when we are improving or reshaping our existing structures. And for new providers it sets out very clearly what the threshold expectations are for sound quality assurance systems.

In a number of places, it is suggested that HEFCE would take a role in defining threshold or baseline requirements. It is actually hard to see how this would be different to what the quality code is supposed to deliver.

3. Shifting the focus of QA from process to outcome

It has always been a criticism of QAA institutional reviews that they look at process not outcomes. This is a fair criticism. However, it is not clear how the five-yearly assurance visits would move this forward; in the consultation document it is made clear that these visits would focus on evidence and processes of the university’s own internal systems.

The suggestion therefore must be that internal systems for QA do not look closely enough at outcomes. This is emphatically not the case: universities have extensive approaches to scrutinise, review and take action based on the data available to measure outputs (e.g. NSS, DLHE, admissions/progression/awarding, OFFA WP agreements, annual reports on complaints and appeals).

4. ‘Strengthening’ the external examiner system

This is an attempt to address an issue that keeps emerging in the sector. We often rightly refer to the external examining system as a fundamental pillar of quality assurance. That does not mean that it is flawless. It is a system of peer review, that makes extensive use of external expertise to scrutinise the standards of student work, assessment and marking practices and the quality of teaching and resources available to students.

The system is very cost effective, as academic staff involved with teaching see it as a core part of their professional practice and so do not charge unrealistic consultancy-style fees for their work. And having been in a position at two universities where I have read almost every external examiner report, I have every faith that externals do not hold back when they have something to say (good, bad or ugly) and academic departments take their views and reports seriously and respond constructively to the recommendations. This is borne out across the sector as evidenced in the reports from QAA’s institutional reviews.

HEFCE’s consultation document has stepped back a little from insisting that something radical will be done – earlier drafts that were circulated hinted at a more extensive overhaul. However, it does still suggest that a national register and training programme is developed for external examiners and that they should coalesce annually to compare standards. I feel certain that this is well-meant, but I am equally certain that if this advice is followed, the external examiner system will be undermined, as universities will struggle to identify and recruit examiners for their programmes.

The system can be finessed, for example it is notable that many external examiners do not feel that are rewarded by their home university for the work they do, but as it stands it is both cost effective, and effective as a significant part of our QA system.

5. Creating a gateway for new providers of higher education to enter the sector

I think we have been inching towards this for some time, but it would be very welcome for there to be a high standard required for new potential providers of HE (as well as active support to help them meet that standard). HEFCE is reflecting what the sector needs here but we will need to see more detail about the plans before making a judgement, and HEFCE may need new powers in order to be able to do it.

The (TEF)lephant in the room

Since this process was started in October 2014, a rather significant thing has happened in the form of a General Election producing a majority Conservative Government with a manifesto pledge to introduce a Teaching Excellence Framework. It’s not yet clear what this will look like and, in a stark contrast to HEFCE’s approach with this consultation, BIS have been reassuringly open, consultative and transparent with the sector about working together to shape this new framework. I think, irrespective of what the TEF might eventually look like, this and the wider QA framework are inseparable because the overlap is going to be significant. Which really does beg of HEFCE the questions: why this new quality system? And why now?

Early conclusions

I think the process that was triggered by HEFCE’s review of quality assessment has broadly come at the right time. There are some serious questions to be asked and answered about where we have now reached with the UK’s arrangements for external QA of universities and HE more widely. It is time to pause, reflect, revisit the fundamental principles of QA and perhaps hit the ‘refresh’ button. The UK system for external QA is admired and replicated around the world – that in itself is not a reason to keep it as it is, but it is a reason to be thorough and reasonable in evaluating its effectiveness.

However instead of a refresh, the review appears to have hit the ‘reset’ button without thinking through the consequences; I fear that if we proceed with these proposals, we may well jettison some bad habits, but we may also put to waste years of good will and lessons learnt.

For what it’s worth, I think it may be wise for some consideration to be given to seeing out the existing cycle of Higher Education Review, then perhaps offering those universities who have been through 3/4/5 cycles of very similar successful QAA review processes a grace period of say 5 years, other less established and new providers would continue to be subject to review in order to maintain scrutiny of their systems. This would allow the TEF and its external review mechanisms to bed in and for the sector and BIS to have real understanding of its impact before moving ahead with another entirely new quality regime.