OfS’ proposals for a more integrated quality system has multiple elements to it, but the one that got the most tongues wagging was the plan for introducing penalties for a Bronze TEF award.
Historically the implication of a Bronze award was that a provider was meeting the expected (high) quality threshold – but under the proposed new system Bronze TEF award holders could expect to be subject to additional regulatory investigation, as well as potentially having to restrict their student number growth. Moreover, the government’s post-16 education and skills white paper indicated that permission to increase fees could also be subject to TEF ratings in future.
Now that we have the outcome of the first round of OfS consultations on the new integrated quality regime, those waiting with bated breath for OfS decision on that particular aspect of the new system will be only partially reassured. OfS has said that it will now no longer consider those holding Bronze awards to be at risk of breaching the B/quality conditions; only those holding a Requires Improvement judgement will be viewed in this light. But Bronze award holders could still be subject to student number limits and reduced eligibility for degree awarding powers or “eligibility for certain funding streams” (such as funding for world-leading specialist providers) with the details on how these will be applied deferred. There will be a second round of consultation on the details of the new quality regime, and also a consultation on the allocation of the Strategic Priorities Grant which will include eligibility for funding based on TEF ratings.
Under the new system providers will actually hold two TEF awards – one for student experience and one for outcomes. The basis for intervention or application of any restrictive measures will be the lower one.
OfS’ goal is to simultaneously avoid the conclusion that a Bronze award represents a failure of quality – ie one requiring punitive regulatory intervention – while at the same time structuring financial and other incentives so as to direct additional funding to those that have been judged to be performing above the threshold. Ultimately, OfS intends that an award of Bronze should indicate baseline compliance with the B conditions while also emphasising that the B conditions articulate what is “adequate” and “sufficient” rather than (the more desirable) “high quality.”
The whole relationship between quality, financial precarity and incentives is all a bit untested – OfS is betting that an incentive and reward system will drive system-wide improvement. There are implicit incoherencies here. Do the medals signal performance at a threshold such that Silver is now the de facto target standard, or do they tell us something about whether providers have evidenced their capacity to achieve continuous enhancement of quality?
There’s also a bit of tension in the notion of incentivising system-wide improvement through depriving those operating at an acceptable level of quality the financial headroom that might make it easier for them to invest in future improvement. But once you’ve committed to the notion of financial incentives – and this is a government agenda, not just OfS – this is where you land. It’s also notable that OfS disapproves of – or considers risky – expansion as a route to financial sustainability, and considers the calculus of expansion as a basis for quality improvement to be “doubtful.”
The new system in a nutshell
So to the (somewhat) less contentious elements of the new system. Going forward, all OfS-registered providers will be subject to a quality (TEF) assessment on a cyclical basis. In the first cycle apprenticeships will be included – with consultation on how to avoid overlap with Ofsted to follow – but postgraduate provision will not be. Franchised provision will be included for both providers, with data provided separately and any material differences in quality affecting ratings.There will be further consultation on achieving closer alignment between TEF assessments and Access and Participation Plans (APPs).
Assessments will consider the student experience and student outcomes. Criteria for the student experience rating will be aligned to conditions of registration B1, B2 and the “effective assessment” element of B4, and assessed via provider submission, and expanded set of NSS indicators, and an independent student submission. Where a provider does not have an established student representative body that would ordinarily undertake the process of convening and independent student submission OfS will expect the provider to facilitate the development of an independent student submission (the keyword here being “independent”) – with further detail in the next round of consultation.
Criteria for the student outcomes rating will be twofold. The first element will be an assessment of whether a provider meets its absolute minimum B3 student outcome thresholds, taking into consideration contextual factors and, additionally, whether a provider can evidence impact from measures taken to improve performance. The current progression measure will no longer feature in this baseline assessment, which will now only consider continuation and completion.
The second element will be a judgement of performance based on a wider range of benchmarked student outcomes, here including progression, students’ judgement of whether they are using what they learned in HE, and salary data, taking into account contextual factors not accounted for in benchmarking and – again – evidence of impact of measures taken to improve performance. There will be further consultation on how to treat “absolute” performance in student outcomes and on how to treat the wider range of benchmarked outcome data.
One change is that there will no longer be an overall rating – providers will hold two separate ratings for student experience and student outcomes. The pool of assessors will be made up of academic experts and student assessors, with further thought to be given to the role of OfS staff in making decisions about the final ratings. Ratings will only be allowed to be challenged for providers rated Bronze or Requires Improvement – no trying to argue your Silver up to a Gold.
One technical point relates to providers whose data is insufficient to draw conclusions – particularly where new or smaller providers have less NSS and student outcomes data to draw on. In this case, rather than trying to find a way to rate those providers regardless, they will simply not receive a rating for any element where the data is insufficient to reach a judgement. Rather than assessing all providers within three years, to allow newer providers to get enough data under their belts, some will be deferred until the first year of the second cycle – with those currently holding Bronze and Requires Improvement in either aspect prioritised for earlier assessment (starting from 2027–28).
Odds and sods
For the wonks, then, a few things we picked up in the consultation response. There’s a continued undertaking to seek to align to European guidelines and standards for quality assurance – an issue dear to the hearts of quality practitioners. Though some respondents felt that efforts to achieve this might increase regulatory cost and burden.
The potential for closer alignment between TEF and APPs is promising, especially to the extent that both TEF and APPs involve an assessment of risks to or prospective gaps in student continuation, completion and progression, and the articulation of evidence-based measures to mitigate those risks or close those gaps. Having the two processes running separately and to different time frames would be a bit of a regulatory own goal, whereas having them work in alignment can only strengthen both.
There is also clearly ongoing disquiet in the sector about the relationship between quality assessment and widening participation – those with more diverse and less advantaged student cohorts observe with some justification that they feel at greater risk of being penalised by the new system. Here OfS reiterates the mantra that all students, whatever their background should benefit from high-quality education, the “motherhood and apple pie” statement of HE regulation. Benchmarking data and the promise to take “contextual factors” into account may go some way to offering reassurance, as does the observation that there are providers holding high TEF ratings who clearly recruit a wide diversity of students. Perhaps this is a tension that will never go away but it deserves continued attention and reflection, and is arguably an area to explore further in the TEF/APP interface work, as well as the technical alignment of the regulatory architecture.
Smaller providers and some FE colleges pointed out that the burden of taking part in TEF might be disproportionate. OfS concedes the point relating to insufficient data but broadly concludes that the burden of quality assessment is one that all providers should accept as part of the cost of doing business. This will disappoint those who might have hoped for a lighter touch version of TEF to be applied.
In the second round of consultation the construction of the range of student outcome indicators will be a major focus. The argument runs that much of students’ post-graduation outcomes is well beyond the control of higher education providers, especially graduate salaries – and also that employment and higher salaries isn;t the only motivation for undertaking HE. The counter argument is that salary, and whether graduates are using what they learned in HE, remains a legitimate proxy for value, which most associate with quality even if the direct link is tenuous. What criteria should be applied to benchmark – especially for specialist providers – is also going to be a bit of a bunfight.
One of the explicit intentions of OfS’ strengthening the expectation of facilitating an independent student submission is to support the development of student representative systems across the sector – in alignment with condition of registration B2 on student engagement.
OfS will publish a “risk monitoring tool” setting out the factors associated with increased risks to quality, which will help to guide decisions about whether interventions might be needed between assessment cycles – the result of which could affect a provider’ TEF rating. Respondents debated whether expansion or contraction of student numbers represents increased risk, what student:staff ratios indicate and whether this should be benchmarked, and the limitations of lag data in giving a timely view of risks. OfS has concluded that rapid increase or decrease in student numbers should be included, that staff:student ratio data will focus on extreme outliers or significant changes, that trend data will be scrutinised as well as absolute data points, and that very low or rapidly declining NSS scores should be an indicator. OfS has also said it will not look at data points in isolation.
If your only question in all of this is “how much will it cost me?” the answer is – something. DfE will no longer be funding OfS to conduct quality assessments, which means that providers have to pick up the tab. OfS will be recalculating the overall costs based on its latest decisions and consulting on whether these should be baked into the overall registration fee – potentially meaning those who get fewer assessments end up paying for those who need more – or levied on a per-assessment basis.
