When it comes to metrics, the TEF is all about outcomes. Of course, it’s all open to consultation, but the painfully deliberate avoidance of input measures (such as academic teaching qualifications) suggests that the consultation is about as open as an oyster with rigor mortis.
Jo Johnson has chosen three outcomes for the ‘common metrics’ that will form the starting assumptions when it comes to institutions vying for TEF banding. The metrics involved are strange choices.
Retention rates are important. (I’ve been banging on about them since the early 90s when Push published institution-by-institution data on drop-out for the first time.) However, research suggests a lack of teaching excellence is rarely the reason for dropping out.
Similarly, satisfaction (NSS scores) are a function of what a student expects as much as what they get. These are both poor proxies for teaching excellence.
But the most poxy proxy is DLHE, the annual survey that monitors students’ progress into jobs six months after graduating.
Leaving aside the many methodological issues with DLHE, the fact of whether someone has got a job is only distantly related to what they have learnt as a result of teaching. If they were the same, it would suggest that when students graduate in a recession and struggle to find work, it must be because their teaching had got worse. That’s obviously not the case.
We shouldn’t be measuring employment. We should measure employability. They’re not the same. When employment falls, such as in a recession, employability often rises as people acquire new knowledge or skills to make themselves more competitive. Employability is the ability to get, keep and succeed in jobs you want – both now and in the future as the economy shifts.
For Mr Johnson, the acquisition of employability should be his ideal HE outcome. Not only does it justify students paying higher fees because their employability has a cash reward, but it also justifies the taxpayer’s contribution to HE as an investment in building a resilient and adaptable labour market.
The problem is that, like any aspect of learning gain, trying to measure employability is like trying to wall-mount jelly – you’re going to need a container.
The Higher Education Policy Institute asked me to write a paper on this subject which is published today as Employability: Degrees of Value. In it, I’ve outlined my choice of jelly container, which is to define employability as having three components.
The first component, knowledge, is at the core of HE and, although degree classifications may not be a perfect way of measuring the transmission of knowledge, that’s not a fight I’m going to pick here.
The second is what I call ‘social capital’, but I admit I’m trying to be provocative. It’s actually wider than that. It’s a whole set of attitudes, behaviours and social and physical attributes – everything from a student’s self-esteem to their ethnicity. I just feel that by calling it social capital, we can recognise that some of this is not about people’s real ability to do jobs. Nor is self-improvement entirely within their control.
This is an area that bears further examination than I had space for in my HEPI paper. Universities neither can – nor should – do much about students’ looks, gender, age or ethnicity, but these are all factors that play a part in their employability, whether rightly or wrongly. Okay, wrongly. We certainly never want to create a perverse incentive for universities to admit students who are more attractive to employers on these unjust criteria.
Then there are attitudes and behaviours that employers do value and, in most cases, are justified in doing so. The uncomfortable truth is that some of these (the ‘right’ accent for high level business, for example, or an anything-is-possible approach) are often associated with social class, again rightly or wrongly. These are some of the reasons many people think independent schools are worth paying money for.
As universities try to weave employability into their courses, they could take a look at which elements of social capital we do want to promote, see how they do it in schools where they get it right and give some thought to doing the same and measuring social capital development in HE too. Co-curricular opportunities and the wider student experience play a big part.
In any case, the role of social capital in employability reminds us that – like so many metrics in HE – it can only really be understood in context.
When the Green Paper tagged wider participation on to the TEF like a wind-sail on a F1 car, it seemed like a well meaning, but misplaced add-on. However, when we think about creating employability within the wider social context, the link begins to become apparent. (Not that I believe that’s what anyone at BIS had in mind).
The third component of employability (the one that features most in my HEPI paper) is skills: ‘soft’ (transferrable) skills and hard (job-specific) skills.
To measure these, Johnson needs to put in place a common (and easy to understand) framework of the diverse range of skills that students are expected to gain from differing courses. The extent to which each skill is acquired would be set against a series of descriptors. Different disciplines and courses would vary, but that’s what we want. People don’t need skills across the board. They need a skill ‘set’ that matches a career.
Students can reflect on their skills at the start and end of their courses and assess themselves, providing a metric of learning gain. More importantly though, the single biggest difference we can make to students’ employability is to raise their awareness of it, increase their understanding of what it’s made of and let them see how well their studies can support their development of skills as well as knowledge.
Similarly, with greater understanding of the components of employability, academics might also benefit and adapt programmes to optimise learning, But the burden must be light: academics must be liberated to teach effectively, pausing only to acknowledge the skills benefits of their courses.
In this way, greater transparency about employability outcomes will help ease the scandal of students taking courses they imagine are career passports when the truth is that the labour market is already flooded in that sector. They’ll see it’s their skills and knowledge that make the difference, not the course label.
After all, the purpose of higher education is not to create a conveyor belt of work-ready drones. It’s about building a civilised society. However, in the process, one of the outcomes has to be that graduates can build careers. Apart from anything else, survey after survey confirms that a getting better jobs in the long term is the single most important reason that prospective students have for going to university.
Read Employability: Degrees of Value here.
Good discussion paper, Johnny. A few thoughts of my own to throw into the melting pot:
1) DLHE should be scrapped now that there exists a mechanism for linking employment records with educational achievement (under the SBEE Act 2015). It’s been far too crude an instrument for decades, even for measuring the effectiveness of a whole system, yet alone individual institutions, and the effort spent collecting it would be better used on improving the curriculum.
2) Any performance measures for employment outcomes must be context sensitive – both in terms of subject, but also in terms of regional and sub-regional economic situation. There are more graduate jobs in London and the South East, so we should not measure the effectiveness of Northern universities by what proportions of their graduates desert the region in which they studied and rush down south (which is what the current data does). Universities and HE are only a part of the education system, so all qualifications should be taken into account and the Learning Records Service used to record those educational outcomes currently missing (HEIs and workplace qualifications) – this will enable more sophisticated statistical analysis, it may be more important in certain jobs what grade you got in GCSE Maths than what University you went to or what subject your degree was in.
3) Government is increasingly running the system based on class prejudices. Employability skills of the sort Johnny describes was a key element of HND courses, now largely abandoned by those universities in England who used to run them (and on a downward spiral). A couple of decades ago the Association of Graduate Recruiters from blue-chip companies produced a profile of the ideal graduate recruit – it corresponded almost exactly with the profile of an HND diplomate from an FE College or Polytechnic! Where did the AGR recruit the bulk of their ‘graduate intake’ from? Oxbridge/red-brick/Russell Group of course! Why were they surprised that they didn’t have the profile they said they wanted? Lip service is paid to the need to expand ‘higher technical education’ of course, but the day a current Tory Minister’s son or daughter qualifies with an HND from their local FE college is the day that we’ll know they mean it. In the meantime it remains what Alison Woolf described it as “A good idea – for other people’s children!”. Universities need to shrug off their indifference and once more work in a genuine partnership with FE to develop and expand vocational sub-degree routes and provision. Maybe the apprenticeship levy, crude as a payroll tax that it is, might be a wake up call.
Yes, really good paper: it makes us think about the value of how we measure all sort of things but also about the value that these things have in relation not only to employment and employability, but also in regard to developing students as future citizens and not just employees. However interesting your three measures are, i would like to add some views of my own.
First of all, you dismiss knowledge too quickly. Knowledge is way more than just the content of what one learns during a course of study. For once, we learn from the core subject we study, but also through our experiences within the University and outside. As a matter of fact that knowledge is the first to be forgotten, bar for the key ideas. Knowledge matters when it is put to some use and i wonder whether employers actually do a good job of that or even understand what is that our students learn. It seems to me that knowledge can be equated to a more traditional view of human capital (and of course there is problem with that).
Social capital is indeed very important, but then i think your social capital does a lot of work. In reality, there are more capitals in your definition, such as moral capital. Social capital, however, is very important because it includes the networks and social connections that each students bring to and leaves with. Some universities can develop and support, and other they cannot (ethnicity and class matter here as either a form of ‘good’ or ‘bad’ credit). Some of the indicators you list, such as behaviours and attitudes, would be better listed under human capital and therefore knowledge would need to be broadened.
finally, skills. If soft skills matter so much, and they do indeed, what are employers doing in helping to develop them? I agree however that universities can do a lot more to provide curricular and extra-curricular activities and opportunities to develop such skills and my university is doing just that.
In conclusion, the time has come to look at what we have done and achieved in the past and find new and better ways to show the value and worth of a university degree.
Some interesting comments but I wouldn’t be so keen to dissociate employability from employment. It is true that, in periods of recession, an individual may in principle increase their absolute level of employability by adding to their stock of human capital through the acquisition of further credentials or the development of skills. However, if we assume that other individuals follow the same strategy then that individual may become no more employable as the overall level of competition for jobs has increased. Unfortunately, there is plenty of evidence from labour market studies of just this pattern of behaviour leading to problems of credential inflation, among other issues.
You’re right of course. I wanted to use a simple example to demonstrate that the two – employment and employability – are distinct from each other. As you suggest, they are related, but the point is that if the end goal is long-term employment and a sufficient labour market, that is more likely to result from a workforce with employability. Meanwhile employability is a reasonable and desirable aim of HE, whereas employment is a metric that invites gaming, unintended consequences and short-term gains.
Surely the problem with all JJ’s metrics is that they are highly correlated with each other, so are in fact measuring more or less one dimension of the ‘success’ Furthermore the variable which links them all is qualification on entry, which itself is largely a proxy for socio-economic status/social capital. We could avoid these measures altogether and determine TEF just by asking students what schools they went to. Crazy though it sounds using JJ measures this is in approximation just what we would be doing.
An excellent point. In the next few weeks, The Bridge Group will be publishing a paper (to which I have contributed) on this theme of translating employability into social mobility through HE. Watch this space.
Very good….. except …..I thought I’d pick up on a couple of points:
1. “They need a skill ‘set’ that matches a career.” – Do they? I thought the whole idea of developing a broad set of graduate attributes was the ability to take all of your learning and apply them in a multitude of careers, and to constantly grow and develop.
2. Hard/Soft Skills -One of the main reasons employers want grads is their ability to apply critical thinking in any given career. Many HE subjects are not vocational, and most students don’t work in an area aligned to their degree. Who could ever imagine (!) a typical 18-21 year old student changing their mind about their future careers either whilst studying or afterwards….
Hard skills will change more than soft skills over time, and consistently employers say they can teach the hard-skills but the soft ones are the pre-requisites. This of-course coupled with extensive lobbying of BIS from industry on the ‘shortcomings’ of graduates whilst industry simultaneously reducing the amount of time and money spent training graduates…..
A really helpful paper – timely corrective to DLHE-fixation. Some thoughts at:
https://headofdepartmentblog.wordpress.com/2016/02/23/what-will-they-do-when-they-grow-up-employability-and-the-academic-experience/