It’s good to talk about mental health, that seems to be a well recognised axiom, but does it follow that it’s good to try and measure it?
We all have different thresholds, different triggers, good days and bad days, but in my experience (and I’m not claiming to be any kind of expert) we also need to be ready and willing to talk. Sometimes, we might need a bit of a nudge, but it’s important that we don’t feel railroaded into opening up, particularly if there’s nobody there to pick up the pieces.
Graduate wellbeing questions
Imagine a scenario where a recent graduate is on the phone to someone in a call centre and the call handler is asking them about their experience of employment since leaving university. They’ve been asked a series of questions about their employment and further study activities and then the caller drops in the following: “Just a couple more questions before you go; firstly, how satisfied are you with your life nowadays?”
For some, such a question wouldn’t be a problem, it’s just a request for information that’s easily dealt with (honestly or otherwise – “I’m fine thanks”). But for others, asked out of the blue with no warning and no context and no support, it’s a trigger, and potentially a destabilising one at that.
This isn’t a farfetched scenario, it’s the context of the Graduate Outcomes survey. And there isn’t just one question, there are four (drawn from the ONS’s personal wellbeing questions). Here they are:
- On a scale of zero (extremely dissatisfied) to ten (extremely satisfied) how satisfied are you with your life nowadays?
- On a scale of zero (not at all worthwhile) to ten (extremely worthwhile), to what extent do you feel the things you do in your life are worthwhile?
- On a scale of zero (extremely unhappy) to ten (extremely happy), how happy did you feel yesterday?
- On a scale of zero (not at all anxious) to ten (extremely anxious), how anxious did you feel yesterday?
Anecdotal problems
However well-intentioned these questions might be, they seem to me to be high risk. I therefore wasn’t surprised to hear from a colleague at another institution who has already had a complaint from a graduate about the wellbeing questions. The graduate wrote:
I had not been warned that I was going to be asked about my mental health… I simply thought I was being asked about my current employment. But after questions about whether or not I was working, I was asked, on a scale of 1 to 10, to rate my satisfaction, my happiness, and my anxiety.
I have been quite upset by this phone call, for reasons which I don’t feel bound to disclose, but I do not think it is acceptable to be called out of the blue and asked questions of this nature, without explaining the nature of the questions, asking permission, and stating what they’re for.”
An entirely foreseeable problem
I’ve since looked back at my response to the New DLHE consultation from nearly three years ago. Here’s what I said about the subjective wellbeing questions:
Subjective wellbeing is certainly of interest, but we feel that the DLHE survey is not the right place to gather it; attempting to do so would risk conflating a number of issues. We are concerned that if the survey employs a telephone methodology, those making calls would be put in a difficult position should the graduate they are interviewing express anxiety raised by the question and want to then discuss it. This is something that the caller has neither the time nor the expertise to do, and dealing with such a situation in these circumstances presents considerable risk.”
A more productive approach to increase understanding of satisfaction would be to try to use data pertaining to graduates’ motivation for entering employment or further study. (Regarding the latter, I wrote about that recently: What about graduate job satisfaction?)
Not only is the gathering of wellbeing information risky, there’s the additional important point recorded in the synthesis of consultation responses on subjective wellbeing (SWB) that “much of SWB is either beyond the power of an HE provider to influence or the effect of the HE experience is difficult to disaggregate from other factors”. So why are we asking them?
Don’t let the anecdotes play out
Currently the problems are only anecdotal, but that doesn’t mean we should wait for this to play out. The problem has been exacerbated by the fact that whereas HESA intended the wellbeing questions to be part of the opt-in question bank, the OfS subsequently made the decision to make them mandatory. I’m guessing this is because the OfS want to be able to gather data to measure progress against their strategic objectives, in particular this one:
All students from all backgrounds, are able to progress into employment, further study, and fulfilling lives”.
However, if my gut feeling highlighted during the consultation period is borne out, and the anecdote above suggests it will be, we shouldn’t be gathering this data at all; just because we can ask something doesn’t mean that we should. And when a) we have no control over the context and b) the respondents who highlight concerns aren’t being offered any support by the call handler, it’s an accident waiting to happen.
Excellent article which makes some very important points. I completely agree with the concerns raised and would urge HESA to review these questions as a matter of urgency.
Excellent piece Stuart highlighting key issues. Metrics only tell us part of the story. The unexpected questions could cause issues post telephone call for the graduate who may dwell on the questions. Ethical considerations need to be considered for any questions asked and survey completers should be given the opportunity to not answer them.
I think you have an interesting point on this and understand the ethical issues it presents. However, it is about a broader conversation of the survey. We need to better understand life after college rather than just employment outcomes. Cannot talk whole university, whole student approach and only focus on one aspect of the life after graduation. The survey then only becomes disingenuous to the necessity of what the university is expected to do, which is employment, well-being, mental health and academic excellence. Sometimes, that isn’t just captured in an employment survey.
I completely agree with the points raised in this article. How are institutions, HESA and the OfS expected to use this data in an effective manner if we have not already established a baseline from before the student leaves their place of study? How will life events impact on the credibility of the data, for example if someone is in an abusive relationship (nothing to do with their University experience) or even if a family member has passed away recently (again, nothing to do with their University experience) – and this is only looking at it from an outside position. If these questions do cause a trigger for the graduates, are HESA and the OfS contributing to the nationwide issue of increasing mental health issues with no aftercare in place? These questions should be removed from an employability survey until there are sufficient answers in place to the questions raised.
Thanks!
Indeed. I’m certainly worried by them
I agree!
I’m glad that this article has been written, I’ve had the same concerns for a long time. I do understand the rationale for asking these questions, but it’s that they will be asked by telephone interviewers with no training in how the questions might make the graduate feel that worries me. If you are an interviewer and someone says that they are 0 on feeling worthwhile and happy, how can you move on to the next question without some sort of acknowledgement of this? And how do you acknowledge this if you’re not trained to? “Well, I suppose that sucks. Anyway, what’s the name of your employer?”.
Also, if the graduate has to explain to a stranger that they are unemployed, and then moments later is asked how worthwhile they think their life is, how is that going to make them feel?
Two other things to think about:
In the consultation, more institutions were against inclusion of subjective wellbeing questions than for them (it was a long time ago but somewhere close to 50% had low or very low support for it).
The OfS have said that they will use the data collected in the first year of graduate outcomes to see the extent to which these measures might vary by subject and provider and therefore might indicate the contribution that providers make to students beyond employment and salary data. A wellbeing league table anyone?!
Agree with what has been said also! If there is no plan to address any well being concerns or sentiments that are expressed by graduates then this seems an odd area to enquire about. After 15 months of life after University I suspect many graduates will be in some way missing golden times, even grieving during such a hard transition – even if the outcome in employment terms is a good one! Am I missing the point of why we need to try and collect data on this?!
That may be the intention, but this still does not obviate the risks to graduate’s wellbeing raised by this article. Either those conducting the GO survey should be equipped to signpost aporopriate support or these questions should be made optional, as was the original intention.
Then stop talking about the whole person. You don’t ask about people’s physical health and have gp’s on campus so why feel the need to ask about mental health and think you can do the job of professional mental health services? You don’t ask if people are married with children and offer relationship advice?
The whole person argument only holds if you ask about the whole person and your not. Your asking about their educational journey, outcomes and mental health.
Health and wellbeing (physical and mental) are hugely important and I get why you want to know but do it with care, understanding and compassion. May I suggest that some national body develop a health and wellbeing survey (maybe even involve some professionals in its development e.g. Mind) that is delivered by people who have been appropriately trained and can arrange support for an individual should they need it. This way the topic would be appropriately introduced and the questions asked in an appropriate way. A way that does more good than harm as opposed to more harm than good.
Thank you
My previous post was intended as a response to Dr Charles
I agree entirely
I am happy that discussions and articles like this are being shown, the graduate experience and mental health is so common needs to be more openly recognized by universities, the struggles not just the successes.
I completely agree with this. I’ve worked as a caller for the DLHE survey and the employment questions alone can be distressing for graduates who have yet to find work or are unhappy in their current roles. To then ask those graduates about whether their life is worthwhile is a terrible idea, compounded by the fact that these calls are usually made by part time student workers who haven’t been trained to deal with mental health and well-being issues.
Has anyone done any research into the impact of this sort of questioning on individuals with mental health issues where there is no follow up support?
At least many (most?) Careers Services offer some level of careers support to their own graduates up to 2 years or more after graduating. It’s not unusual for students and alumni to be dealing with a complex mix of career and mental health issues so Careers Services are used to being on the front line of this, and referring on to other student services where appropriate – but how many universities offer counselling or wellbeing services to their own recent graduates?
I’m not aware of any