Innovations in academic quality assurance come and go. Some have endured, such as the external examiner system, others, like Teaching Quality Assessment (TQA), are much shorter lived. So what are the prospects for the latest quality assurance mechanism on the block, the shiny golden (and silver and bronze) Teaching Excellence Framework?
So what can the history of external examining, Academic Audit (which still exists in a similar state as QAA HE Review), and TQA tell us about whether TEF really has legs?
The external examiner system
The original external control mechanism in higher education is external examining. Perhaps surprisingly given the huge changes which have taken place over the last century, it can be seen as a continuous system, developed and refined over the years but still not that far removed from its original purposes.
External examiners in UK higher education are often viewed as the cornerstone of frameworks for maintaining standards and, by extension, assuring the quality of educational provision. The external examining system as it has existed in the UK since the late 19th century is remarkable for two principal reasons: its resilience in the face of enormous changes and upheaval in higher education, particularly since the early 1990s; and the fact that it remains almost unique among higher education systems in the developed world.
The external examiner system took off properly in the late 19th century but has its origins in the employment of examiners from Oxbridge by the University of Durham in 1832. In addition to Durham’s use of external examiners, Oxford and Cambridge imported examiners from each other and the University of London, from the 1830s, also drew on Oxbridge faculty as externals. They also assisted with the development of the new provincial university colleges which submitted students to London to take external degrees. The charter of the Victoria University at its creation in 1880 provided for examinations to be conducted by internal and external examiners, thereby embedding the system formally in the higher education environment. The model was emulated by other new universities which also drew primarily on examiners from Oxbridge, London, and the ancient Scottish universities before wider extension in the 20th century to professors from other universities and institutions.
Originally, the external examiner was an additional marker, but the role was extended to include responsibilities for the moderation of marks, endorsing the judgements of internal examiners, and confirming the awards made to students. This role has largely been sustained over time with modest adjustments to cope with change, including greatly expanded numbers and different types of degree programme. These developments have created some tension between the traditional responsibilities of external examiners and the more recent requirement to play a greater moderating role and to act as a guarantor of fairness in the treatment of students.
The actual activities undertaken by external examiners tend to include contributions to the setting of examination papers, marking or judging the standard of internal examiners’ marking, and participating in examination board meetings held for the purpose of determining the award of degrees. They may also be asked, or choose, to comment on curricula. At postgraduate level external examiners are involved in examining doctoral theses and masters dissertations.
There are regular calls for overhaul or abandonment of the external examiner system, most recently following further growth in the award of first class degrees. Additionally there are often suggestions that external examining is some kind of cosy club (see for example the comments from Nick Hillman in this BBC piece) where institutions exchange examiners to do each other a favour (as in the mid-19th century). But there is no real evidence to suggest that this actually happens in any significant way. External examining in the UK as it stands now still looks like an extremely efficient and light touch system, but it also serves as something of a firewall against attacks on academic standards as well as a means of defending institutional autonomy, hence the continued widespread support for it within universities.
From external examiners to the CNAA and Academic Audit
Historically, the established universities depended – implicitly and explicitly – on the professionalism and integrity of academic staff, supplemented by the external examiner system, to assure the quality of provision. In the case of pre-1992 public sector higher education (polytechnics and colleges regulated by the CNAA, the Council for National Academic Awards) a raft of external controls was the primary vehicle, but the self-criticality of the academic community was also seen as a vital component. As CNAA began to relax its controls in the late 1980s, permitting greater freedom to the polytechnics to approve their own courses, abolishing the Subject Panels and, ultimately, accrediting institutions (before eventually disappearing after the 1992 FHE Act), the established universities began to put in place procedures and processes which edged them towards the CNAA model.
Beyond the 1986 Reynolds guidelines which were the first such gesture, the Committee of Vice-Chancellors and Principals (CVCP, the forerunner body to Universities UK) established the Academic Audit Unit (AAU) in 1990. The AAU was a belated and (as it proved) inadequate attempt to head off anticipated government intervention in quality assessment.
Academic Audit must have seemed too collegial, relaxed, cosy and far too consensual for government. The reports of Audit visits were not published and there did not appear to be anything – beyond the most modest peer pressure – that was actually holding universities to account for the quality of provision. It is debatable whether anything the CVCP could have done, even assuming it had managed to achieve consensus for a much firmer quality set-up, would have averted the inspectorial thrust of the 1992 Act. The Government at that time was perhaps disinclined to accept anything the universities proposed themselves, probably believing they had had it too free and easy for too long.
The 1992 Further and Higher Education Act
The 1992 Act wrought fundamental change, abolishing the CNAA and allowing the polytechnics to rename themselves universities and to award their own degrees. The Act also created Teaching Quality Assessment (TQA) under the auspices of the funding councils, placing them under a statutory obligation to “secure that provision is made for assessing the quality of education” and led to the separate establishment of the Higher Education Quality Council, putting some distance between the CVCP and the quality assurance function of Audit, and enshrining a modest degree of independence from the universities.
The introduction of TQA, which was to have such a profound impact on universities in the decade which followed, only comprised a few paragraphs of the 1992 Act. TQA was based on the somewhat simplistic notion that the only way to test the quality of provision is to observe the teaching of courses. The funding councils’ interpretation and implementation of the Act delivered a mechanism at the polar opposite of the cosy consensualism of Audit.
Some of the problems with TQA
TQA exemplified many of the major problems and mistaken assumptions about external models of quality assurance. The process was launched in 1992 and its earliest incarnation was not well-worked out, although the funding councils did undertake a small number of pilots. The model adopted in England required all universities offering a particular subject to submit a self-assessment document in which they could make a claim for excellence or self-assess as satisfactory. If a university chose the latter route then it would not receive a visit and the judgement would be accepted by the funding council. If a claim for excellence was made then officers would make a prima facie judgement about whether there were grounds for the claim and, if so, would arrange an assessment visit (and in some cases they could choose to do so anyway even if it was believed that there was not a prima facie case) which could result in an excellent, satisfactory or unsatisfactory rating.
There were very few unsatisfactory ratings awarded during the period of operation of this model (to March 1995), but among the universities there was also considerable discontent about the variability of inadequately specified criteria brought to bear in making judgements. The model placed an overwhelming emphasis on teaching observation but little guidance was provided to TQA assessors on how to reach judgements. Moreover, being rated ‘satisfactory’ turned out to be a very unhelpful student recruitment banner.
From TQA to subject review and then some
The replacement system for TQA – Subject Review – introduced ‘points’ across a range of aspects of provision (six aspects and up to four points in each aspect, making a maximum of 24 points), instead of qualitative overall judgements. It was introduced in the spring of 1995. Although it was never intended that these points be used for ranking purposes in newspaper league tables, HEFCE could not have made it much easier to do so. However, the Welsh and Scottish funding councils retained descriptors rather than moving to a points methodology. A proposed revised method of subject review, due to be introduced in the autumn of 2002, was subsequently abandoned and went back to using descriptors, albeit in modified form.
Both Audit and TQA/subject review continued despite the overlaps and additional burden a dual system presented. They were eventually brought together under the QAA in late 1996 following extended consideration by the Joint Planning Group (the JPG, which did actually suggest dispensing with Audit altogether at one point though this was resisted by the CVCP).
During this period Audit evolved into Continuation Audit (now HE Review) and spawned Overseas Audits and Collaborative Audits too. TQA became Quality Assessment and then evolved into Subject Review from 1995. This amended model also entailed universal rather than selective visits, a reduced emphasis on direct observation of teaching and a much greater stress on documentary evidence. It took nine years (almost ten once a few outstanding delayed reviews had been completed) to complete the first full cycle of TQAs/subject reviews.
We still have the QAA framework, as originally set out by Dearing, to this day. The wide-ranging suite of regulations includes a multi-sectioned code of practice, subject benchmark statements on standards and the national qualifications framework.
Will TEF run and run?
The external examiner system has been in place for over 150 years and is still, despite all challenges, operating effectively. What we now know as HE Review has evolved from its origins as Academic Audit in 1990 through Quality Audit to the current model, but is quite recognisable over a quarter of a century on. TQA, which was modified twice during its operation, had just one full cycle (lasting almost a decade) before it was abandoned.
TEF, like TQA in the 1992 Act, merited only a few references in the Higher Education and Research Act, and yet has already had a far reaching impact on the sector. Like TQA, TEF has three ratings and, although the Gold/Silver/Bronze award mechanism was intended to reassure the outside world that everyone was still a winner, this PIE report on a recent survey of international students suggests it has not quite gone according to plan.
Of the students who claimed to know about the scheme, a quarter think a Bronze award means teaching quality is ‘unsatisfactory’ and half thought results were based on random inspections of lectures and classes from the Department of Education.
It looks a little like the problem of the ‘Unsatisfactory’ rating under TQA is still there. TEF is still in its infancy though, and it is difficult to predict how long will it endure. Many in the sector would be quite happy for it to disappear without trace but, as the recent announcement about Subject Level TEF demonstrates, the Minister is pressing ahead with the next phase of its development. Whilst it is unlikely to outlive external examining, will TEF last as long as Audit and HE Review? Or will it have as short a half-life as TQA and Subject Review? Only time will tell.
