Measurement’s Missing Link: Tracing the Impact of Social Communities

In social or brand community measurement, metrics often relate to things like reach, impressions or followers. Or, as some might say, ‘vanity metrics’ – a term for metrics regarded as holding little value when it comes to making decisions or understanding sales performance.

You might be frustrated by such an accusation. Afterall, you can hardly blame a social (or for that matter any other siloed) team for using the metrics they have available.

Either way, the missing link is whether the efforts to grow those metrics caused an outcome.

Remember the old adage, ‘correlation does not equal causation’; X number of followers does not necessarily equate to Y amount of sales.

Yet as marketers, and at a human level, we inherently understand the value of brand, and brand communities. Of how we engage, shop and buy. Of long-term outcomes. But measurement often fails to adequately capture these nuances.

‍

Direct vs indirect impact

On the one hand, we may hear about the problem of over-inflated ROI from short-term, final touchpoint solutions - such as Retail Media Networks[JG4] , or digital attribution - that take no account of the fact marketing extends across channels, campaigns and time, and rarely operates in isolation. Even more sophisticated measurement systems that aim to attribute over campaigns or touch points, are still often limited to digital advertising, or in-year perspectives.

Which is why you might find paid social teams claiming 5x ROI.

While on the other hand, those building social communities only get to talk about engagement and other ‘soft’ metrics. Things we feel or believe but don’t know.

The trouble is, if you added up all of the ROIs paid teams claim, the sum of the parts might be greater than the whole. And the danger for organic teams is that their work can be thrown out as fluff because they have no idea of ROI.

On both counts, it’s nonsense. Simply, one is ‘measurable’ as a direct impact and the other is not.

But if the effect of those social or brand communities is indirect and hard to measure, how can we be sure that the effort that goes into building them and seeding them with valuable content is worth it?

Making the case for social community measurement

For a brand we work with, social media community members can represent up to 85% of some product unit sales. Consequently, it’s essential to start asking questions to reveal the relationship between that community and sales:

·       Does it help drive long-term sales, or is growth in the social community simply mirroring growth of the business?

·       Do new sales impact the social community or do social communities help prop up sales?

·       Do the communities drive something else that helps drive sales, short or long term?

But many businesses don’t know, or have the tools to answer to these questions. Traditional MMM, even if they’re using it, doesn’t typically measure indirect impacts More broadly, then, you may ask ‘are we overlooking indirect impacts and doing enough to mitigate bias?’.

By deploying an indirect measurement approach, we found that social communities do have an impact on sales, and in what way.

For one client, the social community helped to drive 4.7% of global sell-out within the measurement period, which was proportionally higher than the long-term measurement of paid marketing investment.

Further, growing social community members had the strongest elasticity to structural sales (sales resulting from your promotional efforts or other external factors), followed by the interaction of members with publications. So while member growth had the greatest power to impact sales, to really take advantage of the downstream effects of member growth, rather than relying on organic demand to grow members, it was necessary to combine member growth with a publication strategy.

So while member growth had the greatest power to impact sales, focusing on just member growth wasn’t enough. To drive structural sales, rather than just baseline sales (from organic demand), it was necessary to combine member growth with a publication strategy.

Though brands need to balance their efforts between driving social publication and social community member engagement, because brand publications will saturate over a certain threshold and the impact is then only augmented as the audience size grows.

However, this is only captured in along-term view.

How do you measure indirect impact?

Adding components of indirect impacts, including a long-term view, into your Marketing Mix Modeling (MMM) framework allows you to explore cause and effect

These ‘causal inference’ frameworks help you both to understand the impact and magnitude of the relationships between the community and sales and provide cross-validation of your long-term findings. Meaning you can look for and invest in any positive reinforcement cycles that will drive brand value and engagement to keep community momentum and result in sales.

What’s more, tracing relationships helps you to test hypotheses that are not easily captured in measurements that strictly focus on short-term impacts, or are prone to misattribution from correlation biases. For activities that touch many aspects of the business (e.g. loyalty/app) or are more focused on brand image or engagement (e.g. sponsorship), this can build out a bigger picture.

That means delivering a hybrid solution that is validated and reconciled both through data and expert knowledge to understand these indirect impacts.

A three-step approach to indirect impact measurement

To incorporate indirect impact measurement into your MMM, you will need to focus on structural sales evolution to measure long-term effects while controlling for external factors, test and adopt a causal decomposition module for cross-checking community members’ impact vs external events and use the data to reveal human biases and translate expert business and domain knowledge into mathematical assumptions.

This hybrid approach helps to mitigate correlational bias, builds more robust estimation, and reveals a much richer picture of the impacts by breaking down direct and indirect impacts.

It is executed in three steps:

1. Translate
Translate business knowledge into mathematical assumptions. Refine and adapt your initial graph in collaboration with your social community provider for the specifics of your activity.

2. Estimate
Estimate the social lever impacts on your KPIs. Here is where you can combine causal inference theory with machine learning and sensitivity analysis to obtain the same kind of results as an experimental study, such as A/B testing, using past data.

3. Analyze
Analyze the lever’s impacts on KPIs to draw actionable conclusions. Here you are aiming to aggregate the results to rank and prioritize levers according to the KPIs. Ranking depends upon heterogeneity– how unique it is – and uncertainty.

Fundamentally, if you are going to the effort of cultivating a social or brand community, it is vital to look beyond short-term sales and consider the long-term to be as important in measurement.

Remember:

·      If you do it, measure it and its incrementality

·      Include long-term(structural/baseline) sales as a critical KPI

·      Use causal inference frameworks

By doing so, you will make the measurement links necessary to trace the impact of social communities and provide your organic teams with much missed insight that validates (nor not!) their efforts. No longer dismissed as delivering unmeasurable fluff, not only will they thankyou for it, you will have much richer and more representative measurement of your value-driving marketing ecosystem. And empower better decision-making accordingly.

‍