Previously on Wonkhe I wrote a piece entitled The graduate wellbeing questions are an accident waiting to happen.
To re-cap, the graduate wellbeing questions (or subjective wellbeing questions (SWB), to give them their official term) are as follows:
- On a scale of zero (extremely dissatisfied) to ten (extremely satisfied) how satisfied are you with your life nowadays?
- On a scale of zero (not at all worthwhile) to ten (extremely worthwhile), to what extent do you feel the things you do in your life are worthwhile?
- On a scale of zero (extremely unhappy) to ten (extremely happy), how happy did you feel yesterday?
- On a scale of zero (not at all anxious) to ten (extremely anxious), how anxious did you feel yesterday?
My problem was not with the questions themselves, but rather the context in which they are asked and the lack of support available if asking the questions prompts a negative reaction.
As I concluded in the previous article, when:
a) we have no control over the context and b) the respondents who highlight concerns aren’t being offered any support by the call handler, it’s an accident waiting to happen.”
I was therefore pleased to see HESA wrestling with the issue in a piece on their website recently “asking graduates how they feel”. But whilst it’s good to see HESA engaging thoughtfully with the issues, I was no more reassured by their article – in fact I still have the same concerns.
HESA is doing its best
Perhaps the first thing to note is that this isn’t being driven by HESA but rather by the Office for Students. As HESA points out: “The questions are a core part of the survey at the request of HE regulators”. Secondly, I’m sure that everyone involved in the Graduate Outcomes survey takes the welfare of survey respondents very seriously. But what that means in practice matters.
The questions will be unexpected
Graduate Outcomes is a survey that seeks to “understand whether [graduates are] in employment, have continued with further study or are doing something else and to what extent [their] qualification played a part”. Given the focus of the survey is primarily about employment and further study, questions tagged on at the end about subjective wellbeing lack context and are likely to come as a surprise.
As a result they may prompt a negative reaction in some respondents. The proportion of graduates who may react negatively to these questions is certainly small; HESA have made the questions optional so graduates don’t have to respond, and they currently report that “98.4% answered at least one of the four questions.” Although this sounds reassuring there are two important points in relation to this:
- even though only a small percentage chose not to answer the questions, 1.6% is still a big number (Graduate Outcomes is the biggest annual social survey in the UK – to give you a point of reference the 2016/17 DLHE had nearly 400,000 respondents);
- just because a graduate does answer the questions doesn’t mean that they haven’t had a negative reaction (it could be the answering, as much as the being asked, that prompts a negative response).
There is a distinct lack of support
If a graduate lets a call handler know that the questions (about how satisfied they are with their life and how worthwhile they feel what they do is) have prompted a negative reaction, there is a startling lack of support available. The only thing on offer is “to signpost graduates to the Samaritans if they feel this is appropriate.”
What should be done?
Given that HESA have said that “One probably wouldn’t try to draw causal relationships between higher education experience and wellbeing 15 months after graduation”, I don’t understand why OfS are insisting on keeping the questions as part of the survey.
A much better approach would be to use the very promising looking graduate voice questions as a proxy for wellbeing, which would avoid the risks inherent in asking the subjective wellbeing questions. Questions related to future plans, meaningfulness, and utilising what was learnt provide a really rich dataset that could be extremely useful – and one that the regulator and the league tables should pay much more attention to.
Possibly not intentional but you can read a criticism of the Samaritans from your article – it being the ‘only thing on offer’ and not included in your caveats at the start of the article
Hi Adam, thanks for the comment. On re-reading I can see what you mean, but that isn’t how it was intended at all, apologies. I was simply trying to make the point that potentially prompting a negative reaction and then relying only on a (admittedly brilliant) third party whose area of expertise suicide prevention, is limited. Presumably the majority of those who might be negatively affected would need a very different kind and level of support, and at an earlier stage, and that isn’t provided in this context.
These points are all well-made, Stuart, and it strikes me that, if someone in the corridors of power suspects that many graduates feel anxious, unhappy and dissatisfied with their lives or unconvinced that what they do is worthwhile, then some proper qualitative research might uncover whether or not this is the case. Adding a few simplistic questions to a questionnaire about something entirely unrelated and lacking any context is not the way to go about it!
Thanks for the article Stuart; very interesting but very worrying. OfS May call it “wellbeing” but these are questions about mental health which we all know is a very complex area. To whittle mental health down to 4 questions, completed by self assessment with no mental health backup and support is dangerous. I hope the OfS reconsiders it’s decision to include these questions. There are far better types of questions to ask to see if graduates feel they have direction in life.