Regular followers will know I have some strong feelings towards trend reports. It’s not that they are fundamentally wrong or useless. It’s that they’re used to make decisions, are held up as evidence in arguments, and provide a false sense of security to those that quote them.
For those that don’t know what I’m talking about, trend reports are the weekly or monthly reports we get from footfall providers, survey companies and many other ‘data collectors’. They’re a broad overview of the latest position: what’s changed and how the subject compares to others around the country.
A classic report from footfall camera providers is the weekly summary. It shows that, for instance, ‘footfall is up 5%’ in a particular week, or that the district is doing well against others nationwide. Perhaps it’s bucking the trend.
Cause for celebration? Pull out the champagne bottles? Not quite yet, I’m afraid.
Study the numbers closely enough and it’s pretty clear that footfall, like so many indicators, shifts and bounces as often as the wind changes. There’s rarely a single cause for footfall to go up, or down. City centres are intricately connected, with many tens or hundreds of factors influencing them at the same time.
The weather might’ve caused a surge. Roadworks on an arterial route could be keeping visitors away. The next town has been running a major event. How, then, can we possibly use these reports as direct evidence of our success?
The simple answer is, we can’t. At least – not without drilling much deeper into the figures. If we want to figure out the impact of our work, we need to ascertain how all these factors have played a part in the final tally.
This isn’t easy, but it’s not impossible either. If we have enough data of high enough quality, patterns begin to emerge and we can be more precise in our analyses.
Eventually, after plugging away at the data, we begin to see with tremendous detail. We can get a measure on each and every factor pulling the figures up or down. We can say with ever more confidence that “yes our event was a crowd-puller, and it would have been even better had the roadworks been suspended.”
Before you use trend reports to celebrate success, write a press release, or make that all-important decision, please consider the full picture first. Is that growth the result of your work, or merely another factor at play? Are your statements solid enough to withstand scrutiny at the AGM?
Only by measuring the full impact of all the factors can we figure out exactly what the cause was. This gives us a strong foundation from which to make better decisions – spending more wisely – and evidence to lobby the council or other local groups for change.
This post was originally published on LinkedIn in 2015.