It’s been great to see the Office for Students itself getting in to producing interactive graphs for this year’s National Student survey results. The approach in Nicholson House has been to highlight performance against benchmarks – I’ve taken some other routes.
First up, here’s the numbers that will end up in the league tables and press releases. I’ve long argued that NSS by institution only isn’t helpful for prospective students or others – you include so many different student experiences l that an average doesn’t offer much help for understanding how your experience may compare. Averaging out the experience of fine arts students and nursing students tells us very little.
NSS 2019 – whole institution
My other mainstream NSS bugbear is the focus on “percentage agree” as the gold standard. A while back I took a look at “percentage disagree” – focusing on the ones and twos rather than the fours and fives. This approach has been included in all the visualisations presented here, and in future I think I’ll use it as a default. It takes a lot for a student to go for the lower end of the scale – for me this conveys more meaning about how annoyed students have been than the top end.
You can use the filters to toggle between “registered” (all students on a course run or validated by a provider) and “taught” (all students taught at a provider). I’ve also added the ability to choose which question to look at, or which range (new for 2019). The ranges are as follows:
|Scale 1||The teaching on my course||1-4|
|Scale 2||Learning opportunities||5-7|
|Scale 3||Assessment and feedback||8-11|
|Scale 4||Academic support||12-14|
|Scale 5||Organisation and management||15-17|
|Scale 6||Learning resources||18-20|
|Scale 7||Learning community||21-22|
|Scale 8||Student voice||23-25|
NSS 2019 – by subject
Here I’ve presented the 2019 results in the form that allows you to get as close to results for an individual course as the data allows – using the Common Aggregation Hierarchy Level 3 list of subject. As well as letting you look at “your” subject in “your” institution (or anybody elses), this also gives us the ability to look at average dissatisfaction by subject and level of study. It turns out polymer and textile students are not a happy lot.
NSS – time series
The survey, of course, looks at a different cohort each year, but because we assume cohorts are broadly similar in their attitudes a time series can show changes in institutional performance over time – so here’s the results from 2015 to 2019. The difficulty comes with the changes to the survey between the 2016 and 2017 release – some questions can be mapped across this gap, and others cannot. The default views on this visualisation look at the “overall satisfaction” question (old Q22 and new Q27), but using the filter you can look at any single question or multiple question (I suggest you chose one from the old questions and one from the new).
For the keen, here’s my best attempt at a mapping:
|NEW Qn||OLD Qn||Category||Flag||NEWq||OLDq|
|1||1||The teaching on my course||Unchanged||Staff are good at explaining things*|
|2||2||The teaching on my course||Unchanged||Staff have made the subject interesting*|
|3||The teaching on my course||Scrapped||Staff are enthusiastic about what they are teaching|
|3||4||The teaching on my course||Unchanged||The course is intellectually stimulating*|
|4||The teaching on my course||New||My course has challenged me to achieve my best work [new]|
|5||Learning opportunities [new section]||New||My course has provided me with opportunities to explore ideas or concepts in depth|
|6||Learning opportunities [new section]||New||My course has provided me with opportunities to bring information and ideas together from different topics|
|7||Learning opportunities [new section]||New||My course has provided me with opportunities to apply what I have learnt|
|8||5||Assessment and feedback||Unchanged||The criteria used in marking have been clear in advance*|
|9||6||Assessment and feedback||Changed||Marking and assessment has been fair [amended]||Assesment arrangements and marking have been fair|
|10||7||Assessment and feedback||Changed||Feedback on my work has been timely [amended]||Feeback on my work has been prompt|
|11||8||Assessment and feedback||Changed||I have received helpful comments on my work [amended]||I have received detailed comments on my work|
|9||Assessment and feedback||Scrapped||Feedback on my work has helped me clarify things I didn't understand|
|12||11||Academic support||Unchanged||I have been able to contact staff when I needed to*|
|13||10||Academic support||Changed||I have received sufficient advice and guidance in relation to my course [amended]||I have received sufficient advice and support in relation to my course|
|14||12||Academic support||Changed||Good advice was available when I needed to make study choices on my course [amended]||Good advice was available when I needed to make study choices|
|15||15||Organisation and management||Unchanged||The course is well organised and running smoothly*|
|14||Organisation and management||Scrapped||Any changes in the course or teaching have been communicated effectively|
|16||13||Organisation and management||Changed||The timetable works efficiently for me [amended]||The timetable works efficiently for me as far as my activities are concerned|
|17||14||Organisation and management||Unchanged||Any changes in the course or teaching have been communicated effectively*|
|18||17||Learning resources||Changed||The IT resources and facilities provided have supported my learning well [amended]||I have been able to access general IT resources when I needed to|
|19||16||Learning resources||Changed||The library resources (e.g. books, online services and learning spaces) have supported my learning well [amended]||The library resources and services are good enough for my needs|
|20||18||Learning resources||Changed||I have been able to access course-specific resources (e.g. equipment, facilities, software, collections) when I needed to [amended]||I have been able to access specialised equipment (including computer software, programmes, and rooms) when I needed to.|
|21||Learning community [new section]||New||I feel part of a community of staff and students|
|22||Learning community [new section]||New||I have had the right opportunities to work with other students as part of my course|
|19||Personal development||Scrapped||The course has helped me to present myself with confidence|
|20||Personal development||Scrapped||My communication skills have improved|
|21||Personal development||Scrapped||As a result of the course, I feel confident in tackling unfamiliar problems|
|23||Student voice||New||I have had the right opportunities to provide feedback on my course|
|24||Student voice||New||Staff value students’ views and opinions about the course|
|25||Student voice||New||It is clear how students’ feedback on the course has been acted on|
|26||24||SU||Changed||The students’ union (association or guild) effectively represents students’ academic interests||I am satisfied with the Students' Union (association or guild) at my institution|
|27||22||Overall||Unchanged||Overall, I am satisfied with the quality of the course*|
We should always be aware of the limitations of every survey instrument – both in design (the NSS looks only at final year students, though there are issues faced by part-time students, students on placement years, and non-standard length/format courses that prevents full coverage) and use (the NSS has become increasingly politicised, and you don’t have to look so far to see evidence of incentives to raise the rate of return among engaged students). Looking at the NSS cannot tell us directly about the “quality” of HE provision, but it opens an important window on how it is experienced.
Many responses, therefore, could be said to deal with “hygiene” factors. Issues with access to academic feedback and resources are always a concern in this survey, 2019 is not an exception. And while we should welcome the small average rise in satisfaction (and even more so the small average fall in dissatisfaction, now at the lowest level since 2015) it is the fine grained analysis of the data that will benefit future cohorts.
One response to “Visualising the national student survey 2019”
“polymer and textile students are not a happy lot” : how many are there?
Drilling down should only go so far. Conclusions about 100 may be Ok,
conclusions about 20 (or even less) might fall captive to a circle of friends
on a night out with rarified motives.