I was delivering some training at a students’ union the other day, when one of the questions that came up about the Office for Students’ proposals for the National Student Survey (NSS), its B3 outcomes assessments and the new Teaching Excellence Framework (TEF) exercise was about “belonging”.
“You say that getting the second year has a lot to do with belonging”, started the question. “But does belonging have anything to do with B3 or the TEF?”
If only, I thought, the answer to that question was simple. But it never is, is it?
Surveying the situation
If we take first the National Student Survey (NSS), one of the criteria to focus question design in the the survey suggested by the mid 2010s UK-wide Higher Education Public Information Steering Group (HEPISG) was that it should principally examine the “student academic experience” – a criterion which led to the abandonment of a set of questions on personal development (shunted to the optionals) and a narrowing of the question on SUs to focus their role in academic representation. That proposal was adopted by HEFCE and used to shape the NSS we’ve had since 2017.
It also focuses almost exclusively on “course” – venturing into an institution-wide assessment only when asking about the SU (Q26). “Overall satisfaction” is explicitly anchored to it, and feeling “part of a community of staff and students” is nested in a run of questions that use the word. Questions on stuff like careers, physical environment, welfare resources sustainability are available – but again, they’re in the optional banks.
On an initial reading, nothing in the NSS consultation document indicates that OfS has changed the fundamentals of that approach – in fact when quoting from the HEPISG work on scope in the last decade, the document says:
We are proposing that these criteria should not be amended.
And justifies that approach as follows:
There was a strong desire from providers to retain the academic experience scope of the survey. However, students felt the notion of an academic experience was too narrow – they felt that many factors can influence their overall experience, such as accommodation or local transport issues.
You’ll see there’s something slightly crafty going on there. It is doubtless true that a small number of SU officers and students wanted to see questions in the core on halls or buses. But vastly more will have wanted the survey to look at the overall educational offer from the university – including extracurricular careers and entrepreneurship activity, mental health support, centrally-run study skills activity and SU-organised or funded activities. The document adds:
While acknowledging this wider impact, we take the view that some of these wider factors may be partly, if not wholly, outside the control of a higher education provider. They may also be beyond the remit of one or more of the four UK higher education funding councils and regulators. We are therefore proposing to retain the current criteria.
What’s interesting is that the actual proposed questions go on to break these rules. “How well communicated was information about your university or college’s mental wellbeing support services” is specifically not about the course – ironically it would have been perfectly possible to construct a question on the course’s impact on mental health. And “During your studies, how free did you feel to express your ideas, opinions and beliefs” could be referencing an academic context, or just as easily a time period.
An academic question
Is this an approach reflected in the revised TEF? Fascinatingly, where the NSS consultation uses the phrase “academic experience” throughout, the TEF decision document almost never does – instead deploying the phrase “educational experience” when referring to what’s being evaluated.
But in trying not to sound too “academic” (you know, so as not to exclude “vocational”), it then gets itself confused over education and educational. What was that Mark Twain quote again about never letting your lectures and seminars interfere with your education?
This time OfS references respondents to the consultation that wanted it to incorporate consideration of activities “beyond the educational experience” achieved through, for example, “extracurricular activities such as volunteering”. At this stage you might be thinking “but surely the volunteering activity is educational”, or “but in law aren’t students’ unions established to further the educational purposes of those universities”, but OfS is having none of that:
Our view is that the TEF should be concerned with the student educational experience and the outcomes of that experience, but not the wider higher education experience. We also set out in our consultation that we had considered the recommendations of the independent review and the government’s response to the review, and in particular the emphasis placed on the educational experience throughout the independent review’s findings and recommendations.
That line it is drawing between the “educational” experience and the “wider higher education experience” will come as a surprise to anyone involved in support for extracurricular activities, or those who’ve been designing graduate attribute frameworks or graduate “award” schemes designed to highlight skills development from experiences outside of the bounds of the course.
And anyway, why would an environmental sciences lecturer showing an optional film to the students on their module get TEF brownie points when Envirosoc doing the same wouldn’t? In what way are these skills development modules operated by Coventry SU not educational? And if a university chooses to do some of its education and skills development outside of what it calls a “course”, why would it be penalised?
For the avoidance of doubt… ”
(whenever OfS uses that phrase you know that a dollop of doubt is coming)
…we consider that educational activities may extend beyond the curriculum or the direct delivery of a course and include, for example, academic support or activities that support career development and employability. Such activities would therefore be within the scope of the TEF.
So maybe the film from Envirosoc does count. And maybe being on the committee of Envirosoc does too. Or does it?
We do not think it would be appropriate to assess, for example, experiences that are primarily social, or outcomes or gains that arise primarily from social experiences”.
Fair enough – so a programme of club nights run by the SU during Welcome Week wouldn’t count, but apparently neither would an APP-inspired commuter student breakfast programme designed to strengthen belonging. And a “Black Students’ Leadership development” programme or a workshop from careers on CV-writing would count? Or not? Encouraging those involved in organising activities that aren’t “course teaching”, support for independent study or assessment being encouraged to consider the educational value of that activity sounds great. But is that what OfS means?
Providers may have their own evidence of how some wider activities contribute to the quality of the educational experience or outcomes. Such activities might include, for example, activities that foster a sense of belonging and community, or support for wellbeing….[but] we are not seeking to assess the quality of these activities in the TEF.”
Eh? Isn’t belonging absolutely fundamental to the educational experience? Does this mean that workshops run by careers are out? Given the sector isn’t the NHS, what is university wellbeing support if it isn’t about study? Do programme specific placements count but centrally-run placements don’t? Is it that OfS thinks there are some non-educational activities that are somehow… educational? Make it make sense!
Evidence of how such activities contribute to the quality of the educational experience or outcomes may be considered relevant, if included in a submission… we would expect a provider to clearly demonstrate that a particular activity makes a clear and direct contribution to the quality of the educational experience or outcomes for its mix of students and courses.”
The irony there, of course, is that in many cases departments supporting activities outside of the curriculum can’t get access to data that would demonstrate one way or another whether they’re making a direct contribution – a problem that’s even worse if the data jobsworth is still giving the students’ union the runaround on access to outcomes data for activities participants. It really is time to take a leaf out of some providers’ playbooks and get that fixed.
Access and participation
What’s that you say? Isn’t this all very different to the way in which OfS thinks about access and participation? Funny you should say. Over in APP land, we’re told that the quality of students’ higher education experiences are:
…affected by your organisation’s structures and functions, academic, pastoral and social. With the successful participation and attainment of your students depending on a wide range of factors and experiences, it is important that you take a holistic approach to supporting your students’ success.
So does this mean OfS takes “wider” issues into account when thinking about the experiences and outcomes of underrepresented groups? You betcha!
Effective practice would include consideration of practices across the organisation so that success is supported across academic departments, student services and other departments, as appropriate. This should include engaging colleagues who are not necessarily access and participation specialists such as academics, student support services, senior management, careers services, library services, knowledge exchange professionals and administrative staff.”
And what does that include?
This might include enterprise-related initiatives, including innovation hubs, enterprise modules across the curriculum and business mentors, as well as activities to enhance the contribution that students and graduates can make to their local area and region which can enhance the employability of students and improve successful progression to employment or further study. “
Maybe this all sounds like semantics – but there is something very important here about what we think education is, the activities we value, the sense that students make of their higher education experience and the diversity of providers. Providers and student reps need better clarity than is currently on offer when putting together their TEF submissions, and both applicants and students need information about providers that goes beyond the strict (and alarmingly dated) confines of classrooms, courses and “academic” activity.
Crucially – given everything we’ve researched over the past year and the continued state of the mental health crisis – we also really could do without course leaders and academic societies being told that organising a structured student social isn’t really counted as an “educational activity”. Given the diversity on campus these days, if well done I can’t think of much that is more educational.
