Teamwork and employability – assessing process or outcome?

Those of a certain generation will know well this Morecambe and Wise clip.

André Previn: “You’re playing all the wrong notes.” Eric Morecambe: “I’m playing all the right notes. But not necessarily… in the right order.”

Let’s say we take a set of outcome measures of the piece of music that Eric Morecambe was playing. Number of times each note occurred. Mean durations, with standard deviations. Average volume. Those sorts of things.

It’s perfectly possible (though somewhat implausible, as it happens) for such measures to be exactly the same as those that would derive from the piece he was meant to be playing. We wouldn’t dream of pretending that such measures would be in any way interesting or useful. We would know them to be meaningless.

Assessing outcomes

We see this kind of thing in psychology. Snapshot outcome measures, frozen in a single moment of time, which have stripped away all the temporal and sequential information that gives meaning to the things that people do. It’s rather like adding up the number of times each letter of the alphabet appears in this article and expecting those static, frequency-based measures to capture the sense of the writing.

We see this also in assessment practices. For example, we often want to make judgements about the extent to which our students can work effectively in teams. It’s one of the key skills that every employer will be looking for in a graduate. Yet the measures we use can be as meaningless as those outlined above. Typically, students will be set a project to do in a team. The outcome of that project will be graded. And a pretence is maintained that each student’s competence at teamwork has been assessed, when no such assessment has actually taken place.

Those who have worked in higher education will recognise the scenario. Five awkward looking undergraduates standing at the front of the room doing a group presentation. Each of them talks to a couple of PowerPoint slides. They may end up being awarded a “group grade”, based on the quality of the presentation. Putting to one side the stupendous unfairness of such arrangements (a separate issue in its own right), assessing the outcome of a group project provides no information whatsoever about the effectiveness of each individual’s teamwork skills.

The point is, if you want to assess how well students work together in teams, you have to be prepared to go beyond simple “outcomes”. You have to listen to the music itself. You have to take a look at “process”.

Assessing process

Measures of process can (and do) come in various forms. They might be derived from direct observations made by an assessor. They might come from on-going peer review, from other team members. They may even be derived from the student themselves, perhaps from some kind of logbook that they have been required to maintain. The problem is, such measures are comparatively complex and expensive. They do get used, but mostly as means of mitigating against the worst excesses of the unfairnesses of group grades by enabling adjustments to individual outcomes to be made on the basis of differential abilities and contributions.

To assess an individual’s competence at working together with others, one needs to find ways of assessing the components of teamwork—clarity of expression, effectiveness of listening, ability to include, and so forth—in ways that are somewhat detached from the outcomes of the group endeavour. It should, in theory, be perfectly possible for a student to achieve a first-class teamwork score on a project that only scrapes a third-class pass on a standard outcomes assessment. Indeed, identifying those students who are able to work highly effectively even in relatively unsuccessful groups, may be exactly the kind of information that employers need.

But who among us is competent to assess students in these ways? Sure, we can undertake the difficult task of grading subject knowledge and associated skills, as displayed in essays and reports and presentations and so forth. Academics are employed on the basis that this is one of the kinds of assessments that they are competent to undertake. They are not employed on the basis of their capacity to, for example, make run-time observations of each individual’s contribution to a groupwork exercise and to evaluate the effectiveness of that contribution.

So, if we want to get serious about preparing students for the world of work, we need to get serious about the kinds of assessments that we employ. And there are costs involved.

Solutions

The solution is to ensure that we employ process measures in our assessments when the “thing” that we are assessing demands them. Universities will then be in a better position authentically to assert that their graduates have been able to demonstrate teamworking skills, and students will be in a better position to evaluate their own capabilities in this regard. I’m sure these kinds of assessments are used in the UK HE sector.

But in the subject areas in which I have worked, in the social sciences, they are the exception rather than the rule.

The corollaries of this are twofold. First, some members of course teams must become skilled in using process measures. We don’t need everyone to be able to do this. Indeed, the future of effective assessment depends upon a much more effective deployment of a kaleidoscope of skills that may exist across a subject team, rather than expecting every assessment skill to reside within every member of that team.

Second, in order to create the available resource base for these endeavours, we have to get serious about stripping out the pervasive redundancy in our existing assessment regimes. We conflate the notion of “practice” with the notion of ”assessment”. And we continue to assess the same sets of skills, in the same student, on multiple occasions at each level of study.

Listen to the music

There are major challenges ahead of us, in the HE sector, in preparing students for the world of work. In order to meet those challenges, courses need to resource fewer, more targeted assessments. They also need to get serious about assessing process as well as outcome.