Climate Action Wales Public Engagement Strategy, 2023 to 2026: theory of change (summary)

The Climate Action Wales Public Engagement Strategy (2023 to 2026) (the Strategy) was published by the Welsh Government in July 2023. The Strategy provides a framework for how the Welsh Government and Team Wales will engage with and support people in Wales to act on the climate and nature emergencies. It also aims to ensure that views on the transition to Net Zero are captured across a range of communities, sectors and individuals. In this report, Team Wales, referred to as Partners, includes public service bodies, businesses, voluntary sector organisations and local communities across Wales working together to tackle the shared challenges of the climate and nature emergencies. The purpose of this public engagement is to contribute to the development of green policies that are informed by public dialogue, and accessible to all. The Strategy identifies four green choice themes through which the public can make environmentally sustainable choices: green transport choices, green food choices, green home energy choices and green consumption choices.

Theory-based evaluation will be important for the Welsh Government to understand the impact of the Strategy. As such, the Welsh Government commissioned SYSTRA to develop a Theory of Change (ToC) for the Strategy in May 2024. A ToC is an established evaluation tool that describes what the Strategy aims to achieve and how, by outlining the inputs, activities and outputs necessary for it to achieve the desired outcomes and impacts.

A range of methods were used to inform the development of the ToC. First, SYSTRA conducted an evidence review of relevant literature on the Strategy and facilitated a workshop with central Welsh Government delivery stakeholders. This draft ToC was stress-tested and refined through a series of five further workshops with Welsh Government and external Strategy Partners. Workshop participants gave their input according to their policy area of focus and the corresponding green choice themes within the Strategy.

In addition to developing a ToC, SYSTRA developed guidance for future Strategy evaluation, including an assessment of dependencies, assumptions and risks to the successful implementation of the Strategy.

Below is an overview of the impacts, outcomes, outputs, activities and inputs that were identified for the Strategy (the intervention). Each component is briefly described, followed by a summary of the dependencies, assumptions and risks that were identified through the workshops and the evidence review.

Impacts

The main impact, or the long-term results, the Strategy aims to address is a Just Transition to net zero by 2050. Beyond this, the Strategy aims to contribute to a range of other green and social impacts, including environmental protection, promotion of health, wellbeing and equality across society and in communities, and the promotion of a globally responsible Wales.

None of the impacts will be directly attributable to the Strategy in isolation, so future evaluation must avoid making causal assumptions and consider theory-based and contribution analysis methods. Workshop participants highlighted that solutions to meet social justice needs and achieve net zero may not be delivered due to macroeconomic constraints, for instance that some green lifestyle choices can carry additional cost, and as such, may not be attainable for all.

Outcomes

In a ToC, outcomes refer to the early or medium-term results an intervention aims to achieve. The ToC identifies twelve Strategy outcomes, structured by the COM-B model into three Behaviour change (B) outcomes, ultimately contributing to the Strategy ToC impacts; three Capability (C) outcomes; two Opportunity (O) outcomes; and four Motivation (M) outcomes. In addition, there are 10 engagement (E) outcomes.

The core logic of the Strategy is that the Engagement outcomes are what achieves the Capability, Opportunity and Motivation outcomes (which in turn achieve the Behaviour outcomes). As part of the research, indicators to measure outcomes are suggested in the report. During the workshops, participants raised concerns that opportunity is not sufficiently addressed by the Strategy and there was insufficient underlying policy to enable green behaviour by making it more affordable, desirable and convenient for people. The Strategy also relies on the assumption that involving the public, including disadvantaged groups, in decision-making on the climate will lead to better outcomes for the wider public and disadvantaged groups. However, there is a risk that engagement is conducted but is not effective at achieving its desired outcomes.

Evaluation of the Strategy should assess all elements of the COM-B model included in the Theory of Change outcomes, paying particular attention to whether the Strategy led to increased opportunity for green behaviour change by the public. This will help to minimise the risk of misattributing improvements that would have happened in absence of the Strategy.

Outputs

In a ToC, the outputs refer to what is delivered or produced and should be measurable. The main outputs of the Strategy were consistent and trusted messages, as well as green policy informed by public engagement that is accessible to all. Future evaluation will need to identify appropriate methods for assessing whether the Strategy created these outputs. Some qualitative and quantitative methods for measurement are suggested in the report. Other outputs, especially those relating to engagement and strengthening dialogue and engagement between the Welsh Government and its Partners, may be more challenging to measure objectively. Future evaluation must use a range of methodologies and contextualise the output measurement with the outcomes.

Through the workshops, the risk that campaign evaluation metrics are given too much weight in the early stages of assessing the Strategy was identified. The workshops highlighted that evaluation must consider the interlinked nature of the Strategy’s activities and that all these aspects must be assessed.

Activities

Activities refers to the actions that together constitute an intervention. Underpinning the Strategy is a two-pronged approach to activities, with some delivered by the Welsh Government, and others delivered by Partners. The identified activities were divided into communications, engagement and evidence and social research. These approaches are interrelated, and outcomes of Partner activities should continually feedback to strengthen the Welsh Government activities, and vice versa. This continual feedback and improvement is a core assumption of the Strategy, and that messaging is agreed upon by the Welsh Government and the Partners and delivered consistently. The Strategy’s messaging and communication was informed by research conducted by the Behavioural Insights Team and Beaufort Research using the COM-B model to define ‘Asks’ and ‘Warm-Ups’ to recommend to the public to action sustainable lifestyle changes. An Ask is a recommendation for green behaviour that meets some, but not necessarily all, requirements of the COM-B model, where people are ready and able to change their behaviour. A Warm-Up is a green behaviour that may involve barriers to immediate action, so communications should be designed to increase awareness or offer a more achievable call to action than an immediate behaviour change. An example of a Warm-Up change is to test drive an electric vehicle (EV), which is more accessible to most than buying an EV.

The workshops highlighted some risks to the activities taking place, for instance that not all Partners were able to attribute activities to the Strategy. For example, participants were largely familiar with Wales Climate Week but did not know this was part of the Strategy. Additionally, there was some disagreement on what messaging should be. Some Partners questioned whether it is possible to communicate a consistent message for making sustainable food choices.

The research to develop the ToC was conducted between June and July 2024, around a year after the Strategy had been released. While a mature communications plan was in place all the engagement activities had not yet been conducted. It is likely that further engagement activities for the Strategy will be identified, and the ToC should be updated to incorporate these outputs. Workshop participants identified a risk that messages to the public focusing on encouraging green behaviour might not be supported by the required policy and programmes to make it affordable and convenient for people to implement green lifestyle changes.

Inputs

The resources committed to the intervention, in other words, the inputs to the Strategy, includes resources directly provided or commissioned by the Welsh Government, and expertise provided at a national and international level, for example the Behavioural Science Advisory Group and the Climate Change Committee. In addition, a central input is non-governmental national, regional and local community groups and networks that work on communicating to, and engaging with, regions and communities on the climate and nature emergencies.

Central assumptions to the inputs, identified through the workshops and the evidence review, are that the evidence behind the Strategy is robust and that there are sufficient inputs to meet the requirements of the Strategy.

The following recommendations were made for future evaluation of the Strategy.

Methodological choices

• The Strategy will not directly lead to the outcomes and impacts described in the ToC but aims to contribute to them. Future evaluation should take a theory-based approach, using contribution methods.
• Many of the Strategy’s outcomes can be measured quantitatively, but this would be insufficient to determine whether outcomes and impacts would have happened without the Strategy. Quantitative outcome measurement should be complemented by qualitative assessment to explore how the Strategy led to change.

Considerations relating to dependencies, risks and assumptions

• The Strategy’s activities are interrelated and mutually reinforcing, and future evaluation should be comprehensive rather than assess individual causal links.
• The Strategy is a living document, and some activities are likely yet to be defined, meaning that the ToC will need to be updated.
• The Strategy’s success relies on cooperation between the Welsh Government and its Partners, so future evaluation should incorporate a range of perspectives to understand how the Strategy was implemented.

Report authors: Sally McNamara, Anna Foss

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Jake Wilkinson
Climate and Environment Research
Knowledge and Analytical Services
Welsh Government
Cathays Park
Cardiff
CF10 3NQ
Email: ClimateAndEnvironmentResearch@gov.wales

Social research number: 130/2025
Digital ISBN: 978-1-83715-181-3