Sustainable Management Scheme (SMS): final evaluation (summary)

The SMS aimed to support collaborative landscape-scale projects taking action to deliver nature-based solutions to improve the resilience of our natural resources and ecosystems in a way that delivers benefits to farm businesses and rural communities ^{[footnote 1]}. The SMS provided financial support to 50 projects to undertake a range of activities that would improve the management of Wales’s natural resources and in doing so, contribute to the well-being of rural communities. The scheme offered grants to collaborative groups looking to: 

• enhance biodiversity
• improve green infrastructure 
• sustain better land and water management and
• facilitate climate change adaptation and mitigation at landscape scale ^{[footnote 2]}

SMS projects collaborated with many partners at a local level and at landscape scale to tackle the decline in biodiversity and improve the resilience of ecosystems ^{[footnote 2]}.

The SMS is delivered under sub-Measure 16.5 of the WG RC-RDP 2014-2020 which is funded by the European Agricultural Fund for Rural Development (EAFRD) and the Welsh Government. It is designed to support direct action on managing natural resources across Wales, as part of the Welsh Government’s commitment to sustainable development, as set out in the Well-being of Future Generations (Wales) Act 2015 and the Environment (Wales) Act 2016. The SMS is underpinned by the principles of Sustainable Management of Natural Resources (SMNR) as set out in the Environment (Wales) Act 2016.

Aims and objectives of review

OB3 Research, in conjunction with BRO Partnership, were appointed by the Welsh Government to undertake an evaluation of the Sustainable Management Scheme (SMS)

The aim of the evaluation was to review how the SMS delivered its core aims of supporting collaborative action to improve natural resources and helping achieve ecosystem resilience, and to assess its subsequent contribution to sustaining social and economic benefits for communities.

It was intended that the evaluation explore five main objectives.

1. The alignment of the funded projects with the principles of SMNR ^{[footnote 3]}.
2. The management and implementation of the scheme.
3. The nature and extent of collaboration and what this achieved.
4. The outcomes of the scheme and action on the policy priorities.
5. The contribution projects have made to the cross-cutting themes identified in the Rural Communities-Rural Development Programme, particularly climate adaptation and mitigation.

The evaluation was also expected to review the role of the SMS in contributing to the evidence base for the future delivery of new, post-Brexit agri-environmental funding mechanisms. 

The evaluation was undertaken between January 2020 and March 2023. It has already involved the preparation of a Theory of Change report, a first annual update report in November 2020 and a second annual update report in November 2021.

The final report was prepared in March 2023 as a full outcomes and impact summative evaluation.

This final evaluation stage has involved:

• an inception meeting 
• reviewing relevant policy and legislative documents 
• reviewing scheme monitoring data, the latest available progress reports for each ‘live’ project and the project level evaluation reports received from fourteen completed SMS projects
• drafting discussion guides for interviews with Welsh Government staff, project leads and key partners across SMS projects funded in Windows 3-5 
• developing an online survey for Welsh Government to distribute to all SMS funded projects and analysing 29 responses 
• gathering the views of 43 project leads and key partners across 21 SMS projects funded from Windows 3-5
• interviewing three members of Welsh Government via Teams
• synthesising the findings of the desk-review and fieldwork 
• drafting this final evaluation report

In terms of alignment of the projects to SMNR, the evaluation found that:

• there is strong evidence that SMS projects had considered the principles of SMNR in the design and implementation of their interventions. In analysing how individual projects could evidence a contribution to the nine key SMNR principles it can be concluded that the SMS scheme has aligned well with the approach to managing natural resources and ecosystems outlined in the Environment (Wales) Act in achieving project outcomes 
• there is strong evidence that the overarching scheme design of the SMS has been particularly successful in enabling alignment to some specific SMNR principles – projects are working at a wider scale, are collaborating with new partners or deepening engagement with existing ones and are building increased resilience and longer term thinking into their approaches which results in a wider range of benefits
• there are pockets of examples or emerging evidence too against the remaining SMNR principles and external factors have impacted against projects’ ability to demonstrate progress against some of these as fully as would otherwise be the case
• that a delivery mechanism such as the SMS that funds collaborative action and co-ordinated landscape projects or group activity can successfully tackle the challenges of achieving at scale and deliver multiple actions that are in line with SMNR principles

In terms of management and implementation the evaluation found that:

• at the outset, the SMS scheme stated its intention to establish a more collaborative approach to programme management and from a policy perspective, the design was sound. However, over the evaluation period, projects feel that the visibility of, and their interaction with Welsh Government’s policy team diminished, and many projects described how they have delivered their projects with little or no direct contact with Welsh Government’s policy team. Responsibility for the administration of the scheme was with RPW, and most of the interaction with Welsh Government has therefore been via RPW for claims and monitoring purposes. As a result, the SMS has not been delivered or sustained as originally intended
• with committed spend against funding allocation to the scheme standing at just over half at the end of December 2022, and projects funded during windows 4 and 5 required to deliver within much shorter timescales, the SMS remains at risk of being unable to allocate its total funding in full with substantial underspend of available EU funding as a result
• over the course of the evaluation, projects funded across all windows have raised some serious concerns and frustrations about how administrative issues have impacted on their ability to deliver as planned, and on the negative impact the experience of project managing SMS funded projects has had on individual wellbeing and mental health
• where SMS projects have reported interactions with a member of the SMS policy team, it has been positive and well-received, but those opportunities have been lacking due to inadequate resource in place at a policy team level, which has become ever more pertinent towards the end of the funding of the scheme. SMS funded projects have not reported feeling part of a bigger team, of understanding what other projects are doing or how they are contributing to overarching Welsh Government priorities
• the claims, monitoring and reporting processes adopted via RPW have not been suitable or appropriate mechanisms for the SMS scheme. The design and nature of the scheme, and its intention to fund unique collaborative projects does not necessarily lend itself to be monitored in a simplistic manner. Whilst project outputs are recorded against RDP case indicators, additional outputs achieved by projects are not routinely collected and therefore some important achievements are not being captured
• projects have not submitted independent end-of project evaluation reports in sufficient number or quality. Only fifteen reports were available at the time of drafting this report (of which only four are in addition to those made available for the update evaluation report one year ago), of which less than half are sufficient in their reporting. With such a vacuum of evidence, this scheme-level summative evaluation cannot draw many of the conclusions it would expect to make at this stage
• where collaborative projects have managed to navigate their way through the administrative processes, effective practice has been in place usually because of strong project management and good planning procedures adopted by the lead partner from the outset. In terms of lessons for the future, a more flexible approach to the administration of funded projects, more trust in the robustness of internal processes at well-established charities, organisations and publicly funded authorities would make a huge difference. It remains the case, that until the monitoring requirements and claims processes can be adapted to become more suitable for projects of this nature, the responsibility as a project lead should not be placed on newly established or inexperienced community interest companies or trusts without the support of more established organisations that have the necessary governance and financial viability to deal with payment delay issues

In terms of the nature and extent of collaboration, the evaluation found that: 

• there is evidence that the positive impacts from collaborative working far outweigh the negative for projects particularly in terms of increased transfer of knowledge, more effective delivery at landscape scale and improved engagement with the community. SMS has also shown that collaboration improves over time, and that allowing time for trust and understanding to embed is crucial. The evaluation has not been able to observe any specific typology or type of collaborations that are more likely to succeed although those who put in place strong and regular communication methods, utilised some form of facilitation process to allow all partners to be heard, and who managed to maintain consistency in membership and clarity in terms of responsibility were more likely to succeed
• it is important to allow for additional partners to become involved as projects evolve, and to ensure that key players are around the table
• there are ample examples where collaborative working at the landscape or catchment area between landowners, farmers, and other key partners, has resulted in greater benefits to local nature and ecosystems than had farmers or landowners operated on an individual basis. As a result, we conclude that a collaborative approach to landscape funding should be a key element of government policy in future funding 
• the sustainability of collaborations remains fragile. Across the interviews undertaken, there is evidence of a strong desire to continue to collaborate, but less evidence of concrete plans as to how this will be achieved, without continued funding
• many of the proposed collaborative actions within the SFS will continue to prioritise the actions supported by the SMS. The SFS also seeks to continue with the collaborative approach on a catchment, landscape or national scale and seems to already draw heavily on the collaborative elements of funded SMS projects. It will be important that the learning points from both the design and the delivery of SMS continues to feed across and is integrated into wider environmental and farming policies and funding mechanisms. However, with the transition to SFS some years away, there is a need to design a scheme in the interim that could support the collaborative elements put in place via SMS or new collaborations that have now matured and are ready to deliver
• the approach adopted within SMS could be a key mechanism for delivering some of the objectives set out in COP 15 and for achieving some of the recommendations offered via the Biodiversity Deep Dive. Work at landscape scale must be funded, as farmers and landowners cannot be expected to deliver these ambitions without financial support. The SMS has shown the need for, and could provide the initial basis for, some of the Nature Recovery Exemplar areas operating at landscape scale and with longer term funding.

In terms of outcomes of the SMS, the evaluation found that: 

• it was identified from the outset in the drafting of the Theory of Change report for SMS in 2020 that ‘longer term outcomes set out for the SMS might not be detectable within the timescales set for the evaluation, particularly for most recently funded SMS projects.’ It is therefore unfair to measure the success of the SMS scheme on its ability to demonstrate outcomes that simply cannot be evidenced at this point in time. SMS projects require a longer funding period, and just as importantly a longer reporting period, if they are expected to demonstrate environmental outcomes
• there is evidence of funded projects delivering improvements to natural resources and to the management of landscapes. Activities undertaken by projects report strong outputs that are likely to impact upon improved connectivity between, scale and condition of ecosystems. What cannot be measured or reported upon at this stage is the extent to which these are likely to deliver the intended outcomes in the longer term
• whilst some projects have evidenced sustainable approaches to maintain the improvements made, the majority state that they will be reliant on the goodwill of volunteers, or to successfully obtaining further grant funding to continue with the work undertaken to date. There is a risk, particularly with elements such as peat restoration or removal of INNS that improvements will be lost unless they are protected (e.g., from vandalism) and maintained
• overall, projects are performing well against their wide range of funded KPIs be they environmental, social, or economic outputs. Completed projects have reported strong outputs against their case level indicators, having achieved or exceeded 19 of 24 indicators. The scheme would have benefited from greater consistency in how these indicators were interpreted as well as having in place an outcomes framework from the outset to support monitoring and reporting activities
• most funded projects have taken action which will help to improve natural resources, ecosystems, and landscapes. Projects are confident that their actions will help to increase biodiversity levels and restore habitats, but these anticipated environmental impacts are not expected to be achieved over the short-term funding period. A small number of projects have not yet implemented these type of actions as they are required, as a condition of funding, to complete baseline research and surveys in the first instance to inform their programme of work. In these cases, the environmental benefits are less clear
• projects were commonly found to have taken action which will help to improve the connectivity between ecosystems as they had worked at landscape level to restore hedges across farms and create natural wildlife shelters. Likewise, there are plenty of examples of projects which have taken appropriate action which will lead to improvements in the health and conditions of ecosystems. Actions such as improving water quality, improving soil quality, and removing invasive non-native species can all be expected to improve the health and condition of local ecosystems, provided they are maintained in the long-term. Some of the actions taken require less ongoing maintenance whereas others are more dependent upon continued behavioural changes amongst landowners, farmers, and community volunteers
• projects have made a good contribution towards achieving social benefits for their communities as a result of adopting SMNR principles. The main social benefits generated are increased opportunities for local people to connect with nature, improved health and wellbeing, increased access to the outdoors, and increased educational and volunteering opportunities. The evidence gathered also suggests that the social benefits achieved by SMS projects are more likely to be sustained post funding, particularly those associated with increased access to the outdoors and improving the health and wellbeing of the local community
• by comparison with the social benefits, whilst still positive, projects have made slightly less of an economic contribution to their local communities simply because they did not set out to achieve a wide range of economic outcomes and where they did, the anticipated outcomes were relatively modest. The main economic benefits generated are increased knowledge transfer and the retention of grant funding within local circular economies. Other economic benefits were cited by fewer projects

In terms of EU cross-cutting themes and RDP cross-cutting objectives, the evaluation found that:

• end-of-project evaluation reports have not routinely reported upon the EU cross-cutting themes and RDP cross-cutting objectives therefore conclusions can only be deduced from responses to the survey and interviews. In general, there is evidence across the SMS scheme of equal opportunities being provided to individuals in rural communities to take part in activities. 
• the scheme strongly contributes to the sustainable development objective, with examples of sustainable management of landscapes at its very core. SMS can also demonstrate several very good examples of projects providing free-to-access opportunities for community members to contribute to projects, and to greater spend locally in rural areas. The SMS has also made strong contributions to the RDP cross-cutting activities through its funded activities
• contributions to the Welsh Government’s Welsh language strategy is more mixed. Projects in rural areas with high percentages of Welsh speaking communities, have been highly proactive in the employment of bilingual practices and hosting of bilingual activities to enable individuals to contribute in their language of choice whilst those operating in areas with lower numbers of Welsh speakers have been less likely to do so

To conclude, with the evaluation evidence available to date, SMS projects are certainly able to achieve their deliverables and are performing strongly against their funded outputs and achieving additional benefits too. The overarching collaborative funding mechanism of the SMS has enabled this, but the scheme has not had the appropriate monitoring and evaluation framework in place to capture this evidence as fully as it would have liked. In looking to future funding models, providing projects with the trust and flexibility required to enable them to work in a collaborative, landscape-scale approach that fully aligns with SMNR principles is likely to further improve upon the outcomes that can be achieved.

Based on the evidence gathered and made available to us during the final summative stage of the SMS scheme evaluation, we make the following additional recommendations for the Welsh Government to consider in any future landscape scale collaborative delivery approach.

Recommendation 1

Welsh Government should make sure that the remaining 35 projects fulfil the requirement to provide a high-quality end of project evaluation report and actively promote the guidance developed for this purpose. Some projects have found it difficult to appoint an appropriate external evaluator, and Welsh Government should therefore consider allowing projects to produce their own internal end of project evaluation reports (following the guidance provided). 

Recommendation 2

Welsh Government should explore options to formally capture the outputs and outcomes that have been achieved by completed projects, that are outside of their KPIs. This is not currently possible via RPW Online. Over-achievement should be celebrated, and projects should be encouraged to report upon them in their end of project evaluation reports. 

Recommendation 3

In essence, many of the projects funded in the final two windows of SMS have been ‘demonstrator projects’ as they have not had as much time to deliver. It is also the case that many SMS funded projects have delivered similar activities but are not aware of each other and the approaches taken. Welsh Government should seek to share best practice between projects and ensure that it informs future funding mechanisms. This could be via an end of scheme celebratory event or publication of some kind.

Recommendation 4

There is a gap between the end of the SMS scheme and the proposed development of collaborative action funding under the SFS. We recommend that Welsh Government consider small-scale funding for on-going support for some of the existing partnerships established or developed under SMS. Funding towards a part-time project officer role with the ability to obtain some maintenance funds would be welcomed across projects. 

Recommendation 5

Similarly, a small amount of funding to enable projects to carry out monitoring of progress against environmental outcomes at key points in the future (e.g., in five- and ten-years’ time) would enable the Welsh Government to learn about the benefits achieved from some interventions over a longer period of time. 

Recommendation 6

For any future funding, a development period for partnerships, with appropriate seed money to set up governance arrangements, develop their initial plans and embed collaboration and trust will be important. This could also form part of any interim funding approach so that partnerships are well-established and able to take advantage of future collaborative funding under the SFS. 

Recommendation 7

The SMS scheme has funded several successful models and approaches that have improved collaboration with farmers and landowners, including those led by environmental NGOs and those delivered directly by farmer- led or community-trust led groups. Whilst there is no specific typology that is more successful than others, the common denominator has been strong project management and facilitation that has allowed for all partners to be feel fully involved and informed throughout. We therefore recommend that any future funding mechanism considers how this can be in place throughout the funding period for all projects. 

Recommendation 8

We further recommend that until or unless the monitoring requirements and the claims processes can be adapted to the needs of collaborative projects of this nature, the responsibility for leading projects (and the financial liabilities that sit alongside) should not be placed on newly established or inexperienced community interest companies or trust, without the support of more established organisations so that they are protected from governance and financial viability pressures. 

Recommendation 9

We recommend that the Welsh Government explore a more suitable approach for any future collaborative funding scheme. This could mean making adaptations to the current RPW Online model; establish an Intermediary Body (IM) management model or deliver the funding through another organisation such as the National Lottery Community Fund (NLCF). 

Recommendation 10

Collaboration with key organisations such as NRW, local authorities and Welsh Water has been relatively ad-hoc within the SMS and has been dependent on the interest and contribution of the local officer. A more consistent approach needs to be considered in future. Capturing spatial data and linking to Area Statements and Catchment plans will be key to this and will also enable funded projects to capture GIS data to evidence the impact of their projects in the longer term.

Recommendation 11

Whilst SMS project outputs have been recorded against the appropriate RDP case indicators, additional outputs achieved by projects have not been captured systematically which means that some important achievements have not been reported. Any future collaborative funding scheme needs to establish a detailed outcomes-led framework from the outset that provides flexibility for various projects to report appropriately but establishes clear guidance and definitions, so that outputs and outcomes can be more commonly reported. This will also help ensure that any future funding scheme can report on collective achievements.

[1] Sustainable Management Scheme

[2] Ibid.

[3] Part 1 of the Environment (Wales) Act 2016 sets out a new framework for the sustainable management of our natural resources (SMNR) comprising a definition of SMNR and the nine principles to achieve it and Wales’s Natural Resources Policy (NRP).

Report author: Heledd Bebb and Nia Bryer

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:

Research, Monitoring and Evaluation team
Welsh Government
Cathays Park
Cardiff
CF10 3NQ

Email: Research.Evaluation@gov.wales

Social research number: 42/2026
Digital ISBN: 978-1-83745-126-5