Measuring change is hard. But it is also critical to programs hoping to influence human behaviour towards more positive societal outcomes. In a newly published paper, Does It Work for Biodiversity? Experiences and Challenges in the Evaluation of Social Marketing Campaigns, we tackle the challenge of evaluating social marketing campaigns targeting fishing communities in the Philippines with the goal of driving the adoption of more sustainable fishing practices at the community level.
Research on impact evaluation is vital, particularly in high uncertainty high complexity environmental such as those in which social marketing operates. On by measuring our impact can we first ensure we do no harm and then learn what works, to ensure we improve with each iteration. This is even more pressing in the environmental context, as we have lagged far behind sectors such as public health or international development. Therefore, our goal with this research was showcase how we can raise the bar on the evaluation of behavior change efforts, in this case social marketing, in a particularly changing subject, that of fisheries management in the tropics.
Our research focused on the evaluation of three social marketing campaigns in the Philippines, using a quasi-experiments design of match campaign and control sites. We measured both social indicators through surveys and biological indicators using underwater ecological surveys. We found limited evidence of behaviour change amongst fisherman and no evidence of change in fish biomass as a result of the campaigns. Yet, we also discuss the fact that this last result is fully expected, given how long fisheries take to recover, a timeline often measured in decades, not years. This has implications not only for the way that we plan and implement social marketing campaigns but also for donors who should be aware that expecting biological change in the often short project cycles may be just unrealistic.
Moreover, our research hopes to highlight also the difficulties of carrying out competent impact evaluations in a context where both social and biological indicators need to be measured and where both terrestrial and in-water data is needed. This has obvious implications in terms of cost, not only in terms of money, time and staff but also in terms of required technical expertise. Project budgets need to reflect this reality if we are to be truly evidence based and take responsibility for the interventions we implement. After all it is not about success and failure, it should most of all be about learning.