The first missing numbers: the savings from Universal Credit
This is a post from Missing Numbers, a new blog about the gaps in government data by Anna Powell-Smith. Learn more about this project.
If you’ve read the news in Britain in the past decade, you’ve probably heard of Universal Credit, though you may not be too sure what it is. It’s the current Government’s main welfare policy, which is a fancy way of saying that it is the mechanism through which the government ensures that people who can’t work don’t actually starve or become homeless.
Universal Credit (UC) is a big change from what came before, and many others have written about whether they believe it’s working. But that’s not the point of this blog, Missing Numbers.
Here, I focus on whether the government actually collects the data required to know if this policy is succeeding. What data would the government need, in order to know whether or not UC was working?
Let’s start by looking at the Government’s official aims for the policy. These are:
1. Make the benefit system fairer and more affordable
2. Reduce poverty, worklessness and welfare dependency
3. Reduce levels of fraud and error.
Of course, there’s lots more data it could track to make sure the policy is being delivered successfully, and not having unwanted side-effects - I’ll write about that data in future.
But in short, the official, measurable aims are:
- Reduce the cost of the benefit system
- Reduce unemployment
- Reduce fraud and error.
In this post, I just want to look on point 1: what data the Government is collecting about the savings from UC. (I've been reading Cass Sunstein's excellent book on cost/benefit analysis, which argues that this is the most important data any government can collect.)
What data exists on whether Universal Credit reduces the cost of the benefit system?
In 2012, as UC was first being developed, the Department for Work and Pensions (DWP) laid out its plans for monitoring costs and benefits in its UC Evaluation Framework. It said it would look at five questions:
How much does Universal Cost [sic] to deliver? When do the different benefits and costs occur and what is the difference between the short-run and the long-run steady state costs and benefits? How are the benefits and costs distributed between different groups? What is the monetary value of the various impacts identified under other evaluation themes? What is the overall net value of Universal Credit?
It said these would be tracked via “administrative data”, “programme finance information”, and “outcomes from impact assessments”. But it didn’t go into more detail about what data it would use, or explain how it would use this data to answer the questions.
In 2016, DWP published an updated Evaluation Framework. It did not answer any of the questions above. Instead, on cost/benefit analysis, it said only:
We are continuing to undertake a programme of economic analysis.
And the section on ‘future work’ did not mention cost/benefit analysis at all.
In June 2018, under pressure, DWP made public its Business Case for UC. This estimated that UC would save £8 billion per year once fully rolled out. The Business Case is long, and it would be painful to unpick how data-driven it is, but luckily we don’t have to do that, because the National Audit Office has done it for us.
The National Audit Office's verdict: the data's not strong enough
In its report on Universal Credit in June 2018, the NAO stated the cost/benefit data in the Business Case was not strong enough to draw any firm conclusions:
We cannot be certain that Universal Credit will ever be cheaper to administer than the benefits it replaces… We cannot judge the value for money on the current state of programme management alone.
In the NAO's view, this was because three key data inputs to the cost/benefit calculations were unreliable.
Firstly, because the Government’s savings estimates do not include the extra costs that UC creates for local authorities, which the NAO said DWP does not measure...
The Department... has not sought to systematically collect data on wider costs. It will therefore have no means to assess the full monetary impact that Universal Credit is having.
...and moreover has refused to measure:
The Department has said it will not measure these additional costs until local organisations can demonstrate they are being caused by Universal Credit.
Secondly, because the savings estimates assumed that UC would help many more people find employment - but the data on which this was based is limited and unrepresentative:
It is not known whether the employment impact identified by early evaluation can be replicated across the programme... These studies... covered claimants with relatively simple needs.
(Full Fact has covered this issue more in typically excellent fashion.)
And finally, because DWP's cost/benefit analysis included savings from reduced fraud and error, but the data here also wasn't yet robust:
The Department does not know whether Universal Credit is reducing fraud and error... The Department intends to develop a fully automated risk analysis and intelligence system on fraud and error. But it has not developed this sufficiently to understand and assess fraud and error.
Changes since then
In the House of Commons in October 2018, Ruth George MP asked the Minister for DWP to respond to the NAO’s report:
To ask the Secretary of State for Work and Pensions, what assessment she has made of the effect of the conclusions of the report, Rolling Out Universal Credit, published by the National Audit Office in June 2018...
Minister Alok Sharma supplied a “holding answer”, which is what the Government gives when it does not have time to write a proper answer:
The Government’s conclusion is clear that Universal Credit represents value for money.
It doesn’t look as though the “holding answer” has since been updated with an actual answer.
In October 2018, the Public Accounts Committee recommended (PDF) that the Government should supply better data for its claims:
In future the Department must make sure that all claims for Universal Credit are supported by empirical evidence rather than theoretical models.
The Government's response (PDF) merely said:
The Government agrees with the Committee’s recommendation... The Department intends to regularly update Parliament on progress as data becomes available.
I have searched Hansard, but cannot find any updates (though I did find the DWP minister in February 2019 citing the same employment research that the NAO said was inadequate). Meanwhile, I can find no new published data or research from DWP on cost savings.
So in summary: according to the most authoritative sources I can find, Government either isn't collecting meaningful data on the cost savings of Universal Credit. Or, it is collecting this data but is inexplicably failing to publish it.
Cost savings are the primary stated goal of this programme, and so that makes this officially our first Missing Number.
I need your help
Thanks for reading! I’m really keen to collect more ideas about important missing datasets, on any policy area in any part of the British state. If you’ve got any hints or tips, please drop me a line at anna@anna.ps, or tweet @darkgreener.
Photo credit: Library of Congress