Statistics are great unless they measure the wrong things: Don Pittis

If prices are rising byabout twoper cent, asinflationdata is likely to show this week, why did one of my newspaper subscriptions just go up by 17per cent?

And if wages are rising at about four per cent, as recent jobs data has shown, why are some provincial governments insisting thatwage increases be held below one per cent?

Ashouse prices go through the roof, the fact that the price of the biggest purchase Canadians make in their lives is not included in our inflation statistics makes it easy to see why many young people have expressed doubts about the accuracy of those figures.

It is a struggle that Statistics Canada faces every day as it tries to sketch out with numbers an authenticpicture of the reality Canadians experience. But Oxford fellow and bestselling author of Age of DiscoveryChris Kutarna says the task is far more complicated than many statisticians like to admit.

Kutarnaworries that Statistics Canada'splanto plunge into the oceanof "Big Data" sobeloved ofretailers and credit card companies described last week by chief statistician Anil Arorawill inevitably create bias in the results simply because we are measuring the wrong things.

"One of theterrifying and most fundamental sources of risk is that we only consider what we're now measuring as real," said Kutarna, on the phone from London, England.

For example, long-standing data sets built on debt, spending, prices and gross domestic product simply close the door onvalues such as family, respect, happiness and speciesextinction.

"There is far more that is real and not being measured than there is that is real and we are measuring it," said Kutarna.

One practical example from his book is the failure of modern statistics to measure the value of the online encyclopedia Wikipedia which, despite providing value to billions, adds less to GDP figures than the old Encyclopedia Britannica which reached far fewer people.

In a recent speech, Stephen Poloz, governor of the Bank of Canada, described an economy changing so fast that our statistical models fail to grasp it.

Poloz paraphrased the Solow Paradox, the observation by economist Robert Solow thatcomputers had led to an increase in productivity everywhere but in the statistics. Poloz suggested GDP isbeing understated by as much as twoper cent.

One example he offered was theway so many companies are distributing computer services to the cloud, turning whole computer divisions into a budget line item.

"How does StatCan deal with that?" asked Poloz.

Last week,Aroraboasted to a gathering of the Empire Club that Statistics Canada was respected everywhere as a global leader, but he acknowledgesit is constantly struggling to keep up with changing technology and the shifting understanding ofhow the world works.

"Look, that's what statistics is, right? To take what are evolving concepts, nebulous concepts, things that haven't even taken a lot of shape and then quickly try to turn them into numerics," said Arora in an interview.

The statistics chief calls it a "team sport" where governments and individuals need to decide which of the millions and millions of things that could possibly be measured shouldbe addressed by the some 5,000 employees at Statistics Canada. Their job, he says, is to bring scientific rigour to the process, so that the numbers are as accurate as possible.

"This is always going to be a journey," said Arora, addingthat finding and incorporating into our figureswhat have so far been labelled intangibles may be a never-ending task.

Part of that journey that those employees are now undertaking is the attempt to mine the immense bodies of information embedded in Big Data, thosetraces of activity we leave behind when we do almost anything on the internetfrom buying to searching. Not only are they readily available for quick analysis but they reduce the employee hours required in traditional surveys.

"Alternate sources of data are increasing exponentially and wehave the technologies and the mechanisms to convert them to public good with high quality statistics," said Arorain his speech.

When it comes to the inadequacies of GDP, a big part of the problem has less to do with Statistics Canada than how we continue to use familiar indicators that may be out of date.

Arora says the University of Waterloo's Canadian Index of Wellbeingincludes 200 indicators from crime and safety to sustainable growth most of which come from Statistics Canada data. But it's GDP that gets the attention.

Statisticians are always groping to find the data sets that matter. But even in areas we think we know and understand, statistics are merely an indicator an estimate, of reality. Things we don't understand are, by definition, even harder to measure.

"We live in this culture where what is real is what we measure," saidKutarna, "Thatthe things we measure are reality."

In which case, a certain amount of healthy skepticism, whetherabout this week's inflation numbers,about GDP, productivity or the many other financial statistics that are often offered as solid immutable facts, may well be in order.

Follow Don on Twitter @don_pittis