More students than ever are realising their ambitions to study at postgraduate level. But we still don’t know enough about the quality of their students experience or whether they have successful outcomes.
The Office for Students is committed to ensuring that all students in England have a high-quality experience. A key way to understand that is to hear from students themselves. That is why we are now embarking on a programme to develop options for a future feedback survey for postgraduate taught students.
A quantity of surveys
Since 2005, the National Student Survey has given undergraduates the chance to feed back publicly on their academic experience – to highlight what is working well and what is not. Their insights have helped institutions to improve their teaching. But students in taught postgraduate courses do not currently have the same opportunity, so their voice is not well represented when it comes to strategic thinking in HE or in wider policy development.
There are some existing sources of information. The Postgraduate Taught Experience Survey (PTES) run by AdvanceHE is well established but, by virtue of being focused mainly on teaching quality enhancement, only tells part of the story. As a regulator we also need our own evidence base, with a wider and more comprehensive range of data – telling us what students think about a range of issues, such as whether they are satisfied with the quality of teaching, with their wider experience, and their perceptions of value for money.
First steps
We are beginning with a piece of research. In 2019, we will run a sample survey, inviting all universities and colleges in England to participate by sending it to their postgraduate taught students. This will achieve two things: provide a high-level picture of PGT students’ views about their experience and also to test the questions that we use.
This first stage work will not be published in a way that identifies responses at provider level, and will not be used by the OfS or policymakers to make any judgements about individual courses and providers. Student anonymity will of course be protected.
We are hoping to inform wider feasibility testing for a future larger-scale census survey on the views of PGT students. Of course, this is not a simple prospect. We do not underestimate the complexity of developing a survey that can effectively capture the views of students as diverse as the PGT student body. We will need to test different options – in terms of timing, methodology and content – so that our approach is robust. At the same time we must also be alert to survey fatigue and the need to minimise the administrative burdens.
Later this autumn we will invite postgraduate course providers to take part in this research – and we strongly encourage them to do so. If you represent a university or college, this is a chance not only to gain some valuable information about the views of your students, but to play a role in the development of future approaches to gaining feedback on postgraduate students’ views.
Very interesting, Conor. Thank you for sharing the OfS’s plans. HEFCE had a postgraduate survey as an intention within its 2011-15 business plan but did not manage to achieve that aim, perhaps as a result of having to progress other policy objectives. It was inevitable that the dawning of PGT loans etc would lead to greater focus on the PGT experience from a regulatory point of view. This attention is deserved and welcome.
Let us be clear from the outset, is this survey intended as an accountability measure or a tool that will inform enhancement of the student experience? The answer to that question informs the approach to design; an enhancement tool will require a tremendous degree of research to underpin the psychometric relevance of the questions. The NSS is at least partially credible in this respect.
If OfS is looking to pull together a sector group to advise on this project I’d be very interested!
It seems inevitable that any PGT national survey will end up serving both purposes – accountability and enhancement – so the questions must be robust.
Can we also agree from the outset that whereas it might be convenient to collect responses at one point in the year for all students, that would never work for the PGT population. Instead, if this is to be a true reflection of the total experience, the survey should be triggered as students complete their last piece of assessment (often a substantial dissertation).
The student barometer surveys do in fact survey taught and research graduates. Wheel, reinvention?!
There are three potential uses for a PGT survey of this sort:
1. Accountability – fair enough, that’s OfS’s proper role, but how exactly is this survey going to drive accountability in ways that other PG surveys and systems of accountability do not?
2. Enhancement – PTES already exists as a survey for this purpose. Participation is voluntary, the survey is long and the results are not shared (although you can see your own institution data benchmarked against others). As Bradbury Smith points out, there’s also Student Barometer. Maybe OfS sees a need here for something additional, but given the expense and difficulty of the endeavour, it would have to be a very strong case to persuade me that it’s worth reinventing the wheel, rather than either working with AdvanceHE (which now runs PTES) or iGraduate (Barometer) to adapt an existing product or just working with the existing alternatives.
3. Student choice – a few years ago, I was a consultant on a research project commission by HEFCE to explore postgraduate information needs (see http://webarchive.nationalarchives.gov.uk/20180322111229/http://www.hefce.ac.uk/pubs/rereports/year/2013/pginfoneeds/). The conclusions were pretty clear: an NSS for PGT was not going to answer any general information need for postgrads. They were too disparate a group (eg. demographically diverse, at different career stages, with differing reasons for studying, paying vastly different amounts of money and with varied and individual constraints on their breadth of choice) for a satisfaction survey to be relevant to the individual of whom the sample was highly unlikely to be representative. (There was a concurrent study into the feasibility of a PGT NSS which was no more encouraging.) We did make a number of other recommendations about the information that PGs DO need. Most of those were never thoroughly pursued, including the most important which was that they needed a reliable resource where comparative costs could be researched.
If OfS’s plan is to try again to jolt the PGT NSS corpse back into life for the sake of student choice, I’d really counsel against it. If on the other hand, it’s about accountability, then fair enough, but it’s worth considering whether the same could be achieved through a better use of public money.