Why counting steps doesn’t tell you if someone is healthy…

Steps are measurable, timely and tempting to optimise, but long‑term health depends on genetics, environment, stress, diet, access to care, and time. This is an example of the challenges public sector evaluators face when trying to link medium-term behaviours to long-term outcomes- something which may never be truly solved.

With unprecedented volumes of behavioural data and increasingly sophisticated AI‑enabled analysis, it is tempting to believe that the long‑standing problem of linking behaviour to outcomes in the public sector is on the brink of resolution. If we can now model, connect and predict at scale, surely the gap between what people do today and what governments hope to achieve tomorrow should finally be closing?

Yet in practice, that gap remains stubbornly wide, and may be inherently unbridgeable.

Anyone who has spent time evaluating public sector campaigns will recognise the quiet frustration beneath the rhetoric of progress. Compared to the apparent clarity of commercial measurement, public sector effectiveness can feel maddeningly opaque. Sales appear quickly, are consistently defined, and sit comfortably within a single system of accountability. Public sector outcomes rarely behave this way. Evaluators are instead asked to draw lines between medium‑term behaviours such as applications submitted, services accessed, habits adopted; and outcomes that may take years to emerge, vary radically by context, and are shaped by forces far beyond the reach of any campaign.

This is not a technical problem waiting to be solved by better data. It is a structural one.

The relationship between behaviour and outcomes in the public sector is rarely linear, isolated or provable. It exists at the intersection of behavioural science, policy design and democratic accountability, a space characterised by time lags, political cycles and complex systems. As a result, public sector evaluation has long relied on contribution rather than attribution: assembling plausible narratives about influence rather than claiming causal proof. That distinction is often acknowledged in theory but quietly resisted in practice.

AI has intensified this tension. As modelling becomes more sophisticated, expectations rise accordingly.

Several persistent frictions continue to undermine that ambition.

Time lags are the most obvious. The behaviours governments can observe today often precede the outcomes they care about by years. Medium‑term indicators are therefore treated as leading signals, a pragmatic compromise that allows decisions to be made before outcomes are visible. But signals are not outcomes, and the longer they are relied upon, the greater the risk they are mistaken for success.

Complexity compounds the problem. Public sector outcomes emerge from systems, not interventions. Employment stability, public health or environmental change are shaped by overlapping policies, economic conditions and social norms. Isolating the effect of a single behaviour within that system is rarely credible. This is precisely why contribution narratives persist and why claims of attribution should be treated with caution.

Even where pathways are articulated, they remain fragile. Theories of change increasingly acknowledge non‑linear and interwoven routes from behaviour to outcome.

Data availability only sharpens these risks. Long‑term outcome data is often lagged, incomplete or fragmented across departments, with privacy constraints limiting longitudinal linkage. Faced with these gaps, evaluators gravitate towards what can be measured rather than what most matters. Over time, proxies harden into performance metrics, and performance metrics quietly redefine success.

This is where the real danger lies. When short‑term behaviours are elevated too far, they begin to distort the system they are meant to improve.

None of this is an argument against better data, modelling or AI‑enabled analysis. These tools improve our understanding, narrow uncertainty, and strengthen decision‑making. But they do not remove the need for judgement, nor do they eliminate the political and ethical choices embedded in evaluation.

Perhaps the more uncomfortable question for our industry is this:

For public sector effectiveness, the challenge may not be to finally “prove” impact, but to be more honest about what evaluation can and cannot do. Medium‑term behaviours are valuable signals, but they are not endpoints. Managing that distinction may be the best approach.

Emma Scoular is Executive Director, Effectiveness - OmniGOV at MG OMD. For more on advertising effectiveness, join us at the annual IPA Effectiveness Conference or register for the Advanced Certificate in Effectiveness

The opinions expressed here are those of the authors and were submitted in accordance with the IPA terms and conditions regarding the uploading and contribution of content to the IPA newsletters, IPA website, or other IPA media, and should not be interpreted as representing the opinion of the IPA.