Any marketer can tell you that obtaining reliable, high-quality data is far more difficult than it appears. Survey respondents are notorious for ignoring questionnaires that take longer than ten minutes to complete, while 25% admit to providing random answers to move things along. In 2019, marketers try to work around these challenges by measuring usage statistics and performance indicators, but it’s far from ideal. The most valuable source of information is still the users themselves — if you ask the right questions.
That’s why it’s been fascinating to follow an unfolding analytics experiment in Tulsa, Oklahoma. In 2018, the city partnered with global analytics firm Gallup to launch a “citizen-centered analytics measurement system” that directly captures quality-of-life data points from across the city. This approach holds immense potential for lawmakers and researchers alike, and marketers could learn a few lessons from it as well.
The Gallup-Tulsa CitiVoice Index
The Gallup-Tulsa CitiVoice Index puts forth the results of a representative survey of 4,500 Tulsa residents. It was specifically designed to leverage responses about quality-of-life and civic engagement by measuring a wide range of behavioral factors, as opposed to the standard economic indicators. Each question from the Index is based around one of the following themes:
- Access: Do Tulsans have access to basic needs and services required to lead healthy, productive, and fulfilling lives?
- Opportunity and Growth: Do residents view Tulsa as a destination to live and work?
- Civic Support: How do local institutions, organizations, and Tulsans themselves contribute to improving their local community?
Survey results were broken down by geography and key demographics, along with totals for the entire city. Once Gallup’s analysis was complete, they published their findings to inform policymakers and researchers which performance improvements would most enhance overall quality-of-life.
Why citizen-centered analytics matters
What’s pivotal about the CitiVoice Index is the measurement of outcomes, as opposed to outputs. Traditional research data on urban performance will emphasize crime statistics or building permits, but rarely represent the immediate daily life of actual residents. By placing a focus directly on citizenry, Gallup hoped to find a clearer picture of how elected and appointed city leaders could better serve their constituents.
The difficulty of gathering reliable data is not unique to public surveys — it’s a problem for marketers in almost every field. Even when mobile devices collect massive volumes of data from entire customer groups, analyzing it for actionable insights is challenging. Mobile marketers, for example, typically look to ad spend or retention, which is certainly useful. But as Gallup notes, this is all intended to emphasize outputs, when what we need is the outcome of user experiences.
Consider this informative Hubspot survey, which claims that 83% of respondents may watch automatic video ads, but only 21% have positive reactions to them. The fixed output does not always reflect the goal, forcing marketers to further their analysis. The most important metric we can analyze is user experience, which, though subjective, is still remarkably consistent in outcomes.
This is why websites and mobile apps that emphasize user experience can increase conversation rates by 200% to 400%. It’s why marketers who switched from automatic video to rewarded video ads see higher revenue, retention, and session time. It’s why Airbnb experienced a surge in popularity, and why traditional hospitality brands could be making a comeback.
Using behavioral indicators over output-based metrics
We can see detailed examples just by reviewing the CitiVoice Index. One of Gallup’s goals was to avoid the blind spots created by city leaders who typically rely on “conventional economic indicators.” That’s because while working with fixed numbers such as unemployment rates or housing costs seems reliable, first-hand accounts offer detailed and nuanced insights.
And so, while the CitiVoice Index concluded that Tulsans were doing well overall, it also highlighted many important areas for improvement. Only 50% of residents are classified as thriving, while 46% are struggling and 4% are suffering. At some point in the past year, one-third of citizens found themselves without enough money to buy food — double the national average. Tulsa also discovered that most residents who earn less than $50,000 per year were still classified as thriving if local support organizations were present in their neighborhoods.
Customized solutions vs imperfect policies
Gallup’s emphasis on citizen-centered analytics can benefit residents and policymakers alike by providing Tulsans with a stronger voice in their government. In the past year, only 17% of Tulsans have expressed opinions to city officials, while just 15% attended neighborhood and city council meetings. Forming policies around such limited samples would leave the majority of the population out of key policy decisions.
Since traditional avenues for listening to residents are limited, the Index provided a more comprehensive picture. Its address-based sampling approach gave officials a detailed overview of the entire city, breaking down data points by geography and demographics. Going forward, this will let Tulsa target its solutions to specific regions of the city, rather than implementing city-wide policies that don’t actually help everyone.
Gallup and Tulsa intend to launch new versions of the CitiVoice Index each year, further refining its citizen-centered analytics method. Given Gallup’s pedigree in the field, marketers would do well to watch Tulsa’s quality-of-life improvements in 2020 — and adopt some user-focused lessons for their own work where possible.