We just met the government’s R&D spending target… or did we?

The UK now spends more than 2.4 of GDP on research and development, meeting the government’s target to match the OECD average.

Yet it’s not cause for celebration, as the only thing that’s changed is the measurement. In the real world, nothing has actually changed.

The way we understand the size, growth, and relative success of the economy, and many other facets of life, is through statistics. Good statistics are built on high quality data, which is collected according to agreed definitions and high quality methods.

Sometimes better methods emerge. And sometimes the old methods fail to keep pace with a changing economy. That’s what seems to have happened to the business research and development (R&D) statistics.

The Office for National Statistics (ONS) have just made a huge change to the way they measure business R&D expenditure, increasing estimates by over 50 per cent. This changes our understanding of the recent history of innovation and productivity in the UK. This is a big deal.

R&D is defined by the Frascati Manual – an OECD rulebook, followed by all OECD countries for measuring R&D in their National Accounts. To qualify as R&D, activity must be: novel, creative, uncertain, systematic, and transferable/reproducible

The definition hasn’t changed. But the economy has.

The methodological changes are motivated by a large difference between the amount of R&D implied by claims for R&D tax credits from HMRC, and much lower estimates from the “Business enterprise research and development” (BERD) survey run by ONS. In other countries (according to OECD) survey estimates tend to exceed tax credit records. So either the UK R&D tax credit system has an unusually high uptake, or the UK survey undercounts (or both). ONS clearly think the latter, and so have revised up their survey estimates.

Both the survey and the R&D tax credits use theoretically the same definition (from the Frascati Manual), so why such a huge difference in estimates?

The same definition is likely interpreted differently by survey respondents and tax reporters, given differences in descriptions, guidance, data collection processes, and the people involved. (See my recent paper on the challenges collecting data on intangible assets through business surveys)

And there are clearly different incentives when responding to a survey vs filing for tax credits. This could go both ways: financial benefits from tax credits (and incentive to inflate or claim fraudulently), but also hassle and risk of punishable error.

Underreporting?

We also have to consider whether the sample design of BERD might lead to underreporting.

R&D is (assumed to be) a niche activity – only a small proportion of businesses carry out any R&D. Running the BERD survey as a random sample across the entire business population would be very inefficient, as it would capture a lot of firms for whom it wasn’t relevant, increasing burden for little statistical value.

Instead, BERD samples from a defined population of around 40,000 “known R&D performers”, rather than from the full business population of 2.7m firms. The survey data is scaled up to just the 40,000 firms (with all other 2.66m implicitly given a value of zero). This is unusual.

The BERD survey’s defined population of 40,000 firms is just 1.5 per cent of the total business population of about 2.7 million businesses. Is that a reasonable figure? A selection of other surveys (see below) suggest that the proportion could be much higher, although these surveys cover only larger businesses (of which there are far fewer). The R&D tax credit data covers 84,600 UK companies (of all sizes) in 2020-21 – around double that of the BERD survey.

Notes: All surveys listed cover only firms with 10 or more employees.

Why is the BERD population so low?

Firms are added to the BERD population every year based on information that they are doing R&D. This comes from a range of sources but principally a feeder question on the Annual Business Survey (ABS).

This will likely capture all new performers of R&D amongst large firms, which are likely identified or sampled in one of these surveys every year. However, small firms are infrequently sampled to minimise burden, and likely to fly below the radar for many years.

Crucially, R&D stats are not revised historically for firms found later to be doing R&D – they are scaled to the population from the time, and never revised (until now).

ONS have now made a change to scale up the BERD estimates to account for all the small firms doing R&D that they don’t know about yet. This can be inferred from the ABS, which is a sample survey. For every small business that says they are doing R&D, they are representing many other unsampled businesses that are doing R&D as well. Only the sampled firm gets added to the BERD population, which means the R&D of the unsampled firms remains hidden.

Small and specialist

There is basically no missing R&D amongst large businesses, since they are quickly added to the BERD universe. So the scaling factors are applied by firm size-band, and by type of R&D, since the “missing” share varies.

The result is a huge uplift to the estimates – around £16bn in R&D every year. In England it’s about a 60 per cent increase. In Scotland and Wales, the figures are doubled or more. This completely changes our understanding of R&D in the UK.

In fact, by combining these new business R&D estimates with estimates of R&D by government, universities and non-profits, and the latest GDP data, it suggests the UK has met the government target to spend 2.4 percent of GDP on R&D which was announced in 2017. At the time, the UK was thought to spend just 1.7 per cent of GDP on R&D, and 2.4 per cent was the OECD average.

Is this cause for celebration?

Statistical improvements, of course, are always worth cheering!

Meeting the 2.4 per cent target because we are now measuring it better doesn’t feel like a win: it feels like the target was wrong. That’s no-one’s fault. But we should now revise the target to keep it stretching. The poor productivity performance of the UK remains, and measuring R&D differently doesn’t change that. So we still need to aim for higher R&D spending (and investment in general).

Although, as I set out in a blog for the Economic Statistics Centre of Excellence (ESCoE), Frascati-defined R&D is a narrow measure of innovation investment anyway. We also need to look at other measures, like intellectual property, intangible investment, innovation statistics, and so on.

Some questions remain. ONS says they will be refining their method further, which might involve reviewing the whole survey design. The scale-up approach they’ve used seems a bit crude to me, although it is an improvement. Hopefully ONS will be able to use the R&D tax credit data to inform their sampling and adjustments in future (although that data cannot be a substitute for the rigorous survey-based estimates).

A big question is how far to take any adjustment back, and how to wedge it into historic estimates. Obviously you can’t go back in time and collect more data, so some adjustment will be necessary. Is this a level shift throughout the time series, or has the missing amount of R&D grown over time? At what point historically were BERD stats correct? Since the problem seems to be with missing R&D by small businesses, perhaps the introduction of the SME R&D tax credit in 2000 was the catalyst. Figure 2 illustrates one possible adjusted backseries.

Statistics and data matter – they govern our understanding of the world around us, help us to make decisions, and help government to set policy. As the statistics change, our understanding and policies must change too. The R&D economy is bigger, but only in how we measure it.

The views expressed here are those of the author, and should not be taken as the views of the Bank of England or any of its committees.