Data is vital to planning, measuring, and optimizing full media integration and having enough data can mean the difference between failure and success. After all, data can inform decisions about who is targeted, where the focus is for campaigns, how the audience is engaged, and even what the experience will be like for a consumer.
Insufficient data means that campaigns cannot be measured for results or business impact. This not only harms the potential business impact that the campaigns will deliver, but also often causes companies to reduce investment in these programs due to the inability to measure business impact or ROI.
Moreover, not collecting enough data to get a proper answer is a double whammy in time and money waste. You waste time testing since you spent money on a theory and didn’t even take the time to get good results, and you run the risk of thinking you have a good answer when you don’t, which leads to burning money based on changes from the incorrect answer.
Stephen F. Johnston Jr., founder and CTO at PubWise says if you don’t have time to run proper tests initially and collect enough data, then you should simply go with your gut and run long tests with divergent options.
“Everything about campaign optimization is a balance against ROI,” he says. “You have to spend money to test potentially bad ideas to get a good test and you have to spend time (which is money) on analyzing the test.”
Steve Arentzoff, VP of digital marketing at Cision, says an integrated approach across all media types—paid, owned and earned—is really the key to success.
“Up until now, campaign performance metrics, such as delivery stats, response rates, and revenue attribution were only available to paid and owned channels,” he says. “It is this inability to measure accurate, quantitative campaign statistics that has forced PR and earned media professionals to use vanity metrics for benchmarking their contribution and performance results.”
He explains the lack of statistically valid results data from earned media campaigns imposed an enormous impediment in the ability to manage, measure and optimize earned media campaigns. This gap was not only crucial for measurement, but the lack of results data also prevented usage of standardized performance optimization techniques such as channel testing and creative/message testing.
“If we look at channel and integration, the high trust factor that press releases and earned media campaigns can deliver is a critical enhancement to integrated campaigns, delivering critical credibility that can provide performance boosters to paid and owned channels,” Arentzoff says. “The creation of integrated campaigns across all three media types has created a new opportunity to deliver tighter integration between the digital (paid and owned) and comms marketing disciplines.”
Truly modern CMOs recognize this and are instituting the organizational and campaign planning changes to leverage the opportunities that the new earned media marketing solution has unlocked
campaigns that integrate paid, owned, and earned media channel components have the opportunity to dramatically out-perform silo’ed campaigns that simply target one or two of the media types.
How much is enough?
According to Eric Foutch, managing partner of Red Branch Media, the minimum amount of data truly depends on the campaign timeline. If your campaign is short-lived, say for an event, then week-over-week or month-over-month can provide the best insight for optimization strategies. However, if the campaign has to do with a longer-term goal as in selling a service or solution, you’re not going to want to obsess over the ups and downs of month-over-month data. Instead, focus on analyzing quarterly results as a minimum.
Determining what is enough is inherently a sample size problem and essentially, you need to have enough data from a representative sample of the population. The National Statistical Service has a great calculator for this.
“The primary drivers of sample size are the confidence level and confidence interval,” Johnston says. “With a population of 1 million or more, for a 95% confidence level and a +- 5% confidence interval, you need roughly 390 samples. To get that number to 99% and +-1% is ~16,000 samples.”
Negeen Ghaisar, head of digital strategy, Bigbuzz Marketing Group, says because campaigns for brands can vary significantly, there’s no defined number for any metric, as it varies based on brand, competitor landscape, industry-specific benchmarking and audiences targeted.
“It may also vary depending on seasonality,” she says. “If you don’t understand which ads are working and why, then you don’t have enough data or you’re not doing enough to collect or analyze it, and you should.”
Arentzoff says insufficient data means that campaigns cannot be measured for results or business impact. This not only harms the potential business impact that the campaigns will deliver, but also often causes companies to reduce investment in these programs due to the inability to measure business impact or ROI.
Don’t leave it to chance
Pavel Dmitriev, vice president, data science at Seattle-based Outreach, a sales engagement platform, notes the reason you need more data is that the difference in performance observed should not be due to simply chance.
For example, if emails are sent to a small number of leads, the difference in clicks or replies one sees may be due to chance, yet as they are sent to more and more leads, the difference is less and less likely to be due to chance and more and more likely to be real.
“The actual amount of data needed depends on the amount of improvement we are hoping to detect,” Dmitriev says. “The rule is that the smaller the improvement we care about, the more data we need to confidently measure it.”
The exact number can be calculated for the specific campaign based on the historical data using the process described below. For example, if the current conversion rate is 5%, then the following table give “rule of thumb” numbers:
| Relative change to detect | Required number of emails in each variant |
| 5% | 120,000 |
| 10% | 30,000 |
| 25% | 5,000 |
| 50% | 1,500 |
“The actual formula depends on several other factors in addition to the amount of change we want to detect: the variance in the data, the type I error (acceptable probability of saying there is an improvement when there’s actually no improvement), and type II error (acceptable probability of saying there’s no improvement when there is one),” Dmitriev says. “But the amount of change we want to measure discussed above is the most important factor to think about when deciding how much data to collect.”
The final word
Too little data can be detrimental to any campaign. Lack of data could leave you targeting the wrong audiences, on the wrong platforms and with the wrong creative approach.
Key performance indicators (KPIs) for one brand may be very different for another brand, so to set up an optimal campaign, you need as much data as possible depending on your existing brand’s site, and this often calls for a full research, discovery and testing phase prior to launch.
“I would argue that at minimum, you need as much data as possible depending on your existing brand’s site, platforms used and other potential constraints,” Ghaisar says. “It is essential to understand how users interact with your ads and content with types of engagements, CTRs, CPCs, user flow on your site upon click-through, conversion rates and relevancy. The relevancy part is one I would recommend doing some actual studying and analysis on prior to launch, if budget allows.”
Remember, it might seem tempting to constantly watch a test and stop it as soon as the improvement you see becomes statistically significant, but you should never do that because this increases the chance of making a mistake. Instead you should determine the amount of data to collect in advance and wait until the specified amount is collected before making any decision.