Anthony Hilton: Statistics are lifeblood of democracy, and need to be protected

People rarely think about this but the quality of daily life depends hugely on the quality of statistics.

How much do you pay for your flood insurance? How congested are the roads? How far does your child travel to school? How long do you wait for a hospital appointment? How polluted is the air? How large is your pension? The list goes on.

What governments decide is political; but the areas in which they intervene, the order on which they do things and the way a decision, once made, is implemented, relies on the available data.

But statistics can also reveal truths which politicians would find uncomfortable were they widely known. This is why they have become so adept at distortion — at presenting data in partial ways, usually in a deliberate attempt to mislead so as to maintain or bolster support for their own arguments.

They pay lip service to the need for evidence-led policy; then too often select only the evidence which confirms their prejudices.

Until this month, ministers and departments could even see the stats before anyone else, which in turn allowed them to talk about and emphasise the bits they liked, safe in the knowledge that they could not be challenged because nobody else had the information.

Unfortunately, such behaviour damages the credibility of statistics as a whole. So worried has the UK Statistics Authority become about the effect on public trust that they have now just banned the pre-release of ONS statistics. Thus far ministers have taken it well but it is early days.

There is history here. Government has been known to put pressure on its statistical authority to try to get results more suited to its beliefs, or even to end the collection of certain embarrassing material altogether. Back in the Thatcher era, millions were removed from the official monthly jobless total by making regular small changes in what was defined as unemployment.

Not many people know this but, because the quality of statistics is so crucial, there is in Whitehall an Office for Statistics Regulation, headed by director general Ed Humpherson.

Part of the regulator’s job is to hold government departments to account and get them to back down when they do make claims which are not actually supported by the data. Though these rarely appear in public, this has resulted in some pretty searing letters flying across Whitehall.

But the job is bigger than this because ultimately the quality, trustworthiness and value of our national statistics is the responsibility of the regulatory body.

Statistics are an essential public asset, the lifeblood of democratic debate, and a driver of decisions — not just by government but by businesses, the media, individuals and community groups. So their preparation matters and this is why, last week, the department announced a public consultation on the relaunch and upgrading of the code of practice for official statistics.

The document covers how statistical data should be collected, how its objectivity and reliability is maintained and does not fall victim to vested interests, how the data can be used and how those doing the work should be protected from external influences. It is a guide to how people should work and a shield to protect them while they do.

At its core it is about defending the integrity of those who do the work, and maintaining the integrity of what they do, being honest about the limitations of statistics and not pretending they are what they are not.

The fundamental purpose is to produce robust and reliable data which is also consistent and relevant to society. It is about creating an organisation which acts transparently and in the public interest while respecting the privacy of all the individuals and businesses. As such its guidance has applications well beyond Whitehall and could usefully be applied to private-sector data collection too.

There is also a wider issue which again affects us all whether we understand it or not. The old way was where statisticians sourced and collected the data on which they worked — they sent out surveys and analysed the responses.

What has changed in recent times, however, is that the quality and speed of data collection by the private sector has grown out of all recognition.

Facebook, Google and Amazon obviously have huge amounts of information on their users and customers, but so too does Tesco via its store cards and logistics firms which can track parcels and taxi companies which know who is going where. What is called big data can deliver information about patterns which have never been obvious before.

The mining of this information to give insights into behaviours which can be turned to commercial advantage is now a major pre-occupation of business and one of the things which is going to define success and failure in future.

But this also significantly widens the pool of data potentially open to use by the Office for National Statistics so it is vital that equivalent standards of quality and integrity apply if trust is to be maintained.

So, statistics are as vital to society as all those other bits of infrastructure — water, electricity, broadband, sewers — that we take for granted and never think about until they don’t work. What goes wrong with statistics is that people lose trust in them.

What the regulator’s office is trying to do is show that truth does still matter. We have had the fake news and the rubbishing of experts and can already see that society does not function well in such circumstances. Humpherson with his code is part of the fightback.