Visualising the national student survey 2019

The National Student Survey generates a lot of data - David Kernohan helps you sift through it.
This article is more than 1 year old

It’s been great to see the Office for Students itself getting in to producing interactive graphs for this year’s National Student survey results. The approach in Nicholson House has been to highlight performance against benchmarks – I’ve taken some other routes.

First up, here’s the numbers that will end up in the league tables and press releases. I’ve long argued that NSS by institution only isn’t helpful for prospective students or others – you include so many different student experiences l that an average doesn’t offer much help for understanding how your experience may compare. Averaging out the experience of fine arts students and nursing students tells us very little.

NSS 2019 – whole institution

[Full screen]

My other mainstream NSS bugbear is the focus on “percentage agree” as the gold standard. A while back I took a look at “percentage disagree” – focusing on the ones and twos rather than the fours and fives. This approach has been included in all the visualisations presented here, and in future I think I’ll use it as a default. It takes a lot for a student to go for the lower end of the scale – for me this conveys more meaning about how annoyed students have been than the top end.

You can use the filters to toggle between “registered” (all students on a course run or validated by a provider) and “taught” (all students taught at a provider). I’ve also added the ability to choose which question to look at, or which range (new for 2019). The ranges are as follows:

Scale 1The teaching on my course1-4
Scale 2Learning opportunities5-7
Scale 3Assessment and feedback8-11
Scale 4Academic support12-14
Scale 5Organisation and management15-17
Scale 6Learning resources18-20
Scale 7Learning community21-22
Scale 8Student voice23-25
Students' union26
Overall satisfaction27

NSS 2019 – by subject

[Full screen]

Here I’ve presented the 2019 results in the form that allows you to get as close to results for an individual course as the data allows – using the Common Aggregation Hierarchy Level 3 list of subject. As well as letting you look at “your” subject in “your” institution (or anybody elses), this also gives us the ability to look at average dissatisfaction by subject and level of study. It turns out polymer and textile students are not a happy lot.

NSS – time series

[Full screen]

The survey, of course, looks at a different cohort each year, but because we assume cohorts are broadly similar in their attitudes a time series can show changes in institutional performance over time – so here’s the results from 2015 to 2019.  The difficulty comes with the changes to the survey between the 2016 and 2017 release – some questions can be mapped across this gap, and others cannot. The default views on this visualisation look at the “overall satisfaction” question (old Q22 and new Q27), but using the filter you can look at any single question or multiple question (I suggest you chose one from the old questions and one from the new).

For the keen, here’s my best attempt at a mapping:

NEW QnOLD QnCategoryFlagNEWqOLDq
11The teaching on my courseUnchangedStaff are good at explaining things*
22The teaching on my courseUnchangedStaff have made the subject interesting*
3The teaching on my courseScrappedStaff are enthusiastic about what they are teaching
34The teaching on my courseUnchangedThe course is intellectually stimulating*
4The teaching on my courseNewMy course has challenged me to achieve my best work [new]
5Learning opportunities [new section]NewMy course has provided me with opportunities to explore ideas or concepts in depth
6Learning opportunities [new section]NewMy course has provided me with opportunities to bring information and ideas together from different topics
7Learning opportunities [new section]NewMy course has provided me with opportunities to apply what I have learnt
85Assessment and feedbackUnchangedThe criteria used in marking have been clear in advance*
96Assessment and feedbackChangedMarking and assessment has been fair [amended]Assesment arrangements and marking have been fair
107Assessment and feedbackChangedFeedback on my work has been timely [amended]Feeback on my work has been prompt
118Assessment and feedbackChangedI have received helpful comments on my work [amended]I have received detailed comments on my work
9Assessment and feedbackScrappedFeedback on my work has helped me clarify things I didn't understand
1211Academic supportUnchangedI have been able to contact staff when I needed to*
1310Academic supportChangedI have received sufficient advice and guidance in relation to my course [amended]I have received sufficient advice and support in relation to my course
1412Academic supportChangedGood advice was available when I needed to make study choices on my course [amended]Good advice was available when I needed to make study choices
1515Organisation and managementUnchangedThe course is well organised and running smoothly*
14Organisation and managementScrappedAny changes in the course or teaching have been communicated effectively
1613Organisation and managementChangedThe timetable works efficiently for me [amended]The timetable works efficiently for me as far as my activities are concerned
1714Organisation and managementUnchangedAny changes in the course or teaching have been communicated effectively*
1817Learning resourcesChangedThe IT resources and facilities provided have supported my learning well [amended]I have been able to access general IT resources when I needed to
1916Learning resourcesChangedThe library resources (e.g. books, online services and learning spaces) have supported my learning well [amended]The library resources and services are good enough for my needs
2018Learning resourcesChangedI have been able to access course-specific resources (e.g. equipment, facilities, software, collections) when I needed to [amended]I have been able to access specialised equipment (including computer software, programmes, and rooms) when I needed to.
21Learning community [new section]NewI feel part of a community of staff and students
22Learning community [new section]NewI have had the right opportunities to work with other students as part of my course
19Personal developmentScrappedThe course has helped me to present myself with confidence
20Personal developmentScrappedMy communication skills have improved
21Personal developmentScrappedAs a result of the course, I feel confident in tackling unfamiliar problems
23Student voiceNewI have had the right opportunities to provide feedback on my course
24Student voiceNewStaff value students’ views and opinions about the course
25Student voiceNewIt is clear how students’ feedback on the course has been acted on
2624SUChangedThe students’ union (association or guild) effectively represents students’ academic interestsI am satisfied with the Students' Union (association or guild) at my institution
2722OverallUnchangedOverall, I am satisfied with the quality of the course*

Other thoughts

We should always be aware of the limitations of every survey instrument – both in design (the NSS looks only at final year students, though there are issues faced by part-time students, students on placement years, and non-standard length/format courses that prevents full coverage) and use (the NSS has become increasingly politicised, and you don’t have to look so far to see evidence of incentives to raise the rate of return among engaged students). Looking at the NSS cannot tell us directly about the “quality” of HE provision, but it opens an important window on how it is experienced.

Many responses, therefore, could be said to deal with “hygiene” factors. Issues with access to academic feedback and resources are always a concern in this survey, 2019 is not an exception. And while we should welcome the small average rise in satisfaction (and even more so the small average fall in dissatisfaction, now at the lowest level since 2015) it is the fine grained analysis of the data that will benefit future cohorts.

One response to “Visualising the national student survey 2019

  1. “polymer and textile students are not a happy lot” : how many are there?
    Drilling down should only go so far. Conclusions about 100 may be Ok,
    conclusions about 20 (or even less) might fall captive to a circle of friends
    on a night out with rarified motives.

Leave a Reply