Evaluation of Business Support Services Wales (BSSW): summary

In March 2022 the Welsh Government commissioned Wavehill to undertake a final evaluation of the Business Support Services in Wales programme and, in particular, the operations delivered under the Business Wales brand, as well as the Superfast Broadband Exploitation operation. All operations commenced in 2016 and continued delivering support until 30th April 2023.

The operations have been delivered via the 2014 to 2020 European Structural Funds programme through the European Regional Development Fund (ERDF) across the two operational programmes in Wales, namely West Wales and the Valleys (WWV) and East Wales (EW). The operations have been funded through Priority Axis 2: Small and Medium-sized Enterprises (SME) Competitiveness under the following Specific Objectives:

• SO2.2: To increase the number of SME start-ups through the provision of information, advice and guidance and support for entrepreneurship;
• SO2.3: To increase the take-up and exploitation of next generation access (NGA) networks and ICT infrastructure by SMEs;
• SO2.4: To increase the growth of those SMEs with growth potential, in particular through accessing new markets (both domestic and international).

The six operations funded under these SOs that fall within the scope of the evaluation are set out below:

• Entrepreneurship Support – Business Wales (operations in EW & WWV)
• Superfast Broadband Exploitation (EW & WWV)
• SME Support – Business Wales (EW & WWV)

The services delivered under these operations were:

• Superfast Broadband Exploitation (SFBE) which supported businesses to take advantage of technology and broadband to improve business performance.
• Business Wales Core and Growth (Levels 1–4) Service which supported Welsh entrepreneurs and SMEs by offering access to information, workshops, online and face-to-face advice, specialist support, start-up support, and general business advice.
• Business Wales Accelerated Growth (Level 5) Programme - which offered bespoke support for rapidly growing high-growth pre-revenue and small and medium-sized businesses.
• Business Wales Enterprise Hubs (WWV only) which offered ERDF funding for the support for the running of three of the five Business Wales Enterprise Hubs throughout Wales with ERDF funding supporting those Hubs based in WWV. 

This is a final evaluation that builds on a mid-term evaluation^{[footnote 1]} completed in 2018. The final evaluation has involved the following approaches:

• A review of key documentation to provide contextual understanding of the origin and design of the operations and a sense of performance against targets over time.
• Analysis of management information (MI) held by Welsh Government to explore patterns of engagement and support. The MI data was then matched to evidence from recipients of support from the above operations, captured through a beneficiary survey undertaken as part of a separate evaluation of ERDF support for business^[footnote2].
• Six scoping interviews undertaken in August to September 2022 to capture further insight into the operations and the experiences of delivery. A further 19 stakeholder consultations were undertaken in March to April 2023. Stakeholders were those who could provide an informed, strategic perspective on the provision or who had been involved in the delivery of the support services associated with the operations.
• A Counterfactual impact evaluation (CIE) was conducted to explore firm-level employment and turnover impacts amongst participant businesses. 

Methodological limitations

The data matching process was undermined by a combination of inaccurate and incomplete company registration numbers, it was therefore not possible to match 22% of participant businesses.

Whilst the operations commenced in 2016, the sample frame for the survey placed emphasis on surveying participant businesses who had engaged with the support from March 2019 onwards to strengthen the recall of business support amongst respondent businesses. However, 78% of Business Wales (BW) Accelerated Growth Programme (AGP) clients enrolled in the service prior to March 2019 (compared to 40% of Level 1-4 clients) and were therefore excluded from the sample frame. This information was unknown to the evaluators at the point of survey and sample frame design. 

The CIE approach used company registration numbers (CRNs) as the unique identifier for matching data however only 5.4% of those in receipt of Entrepreneurship Support included CRNs. Therefore, the impact for participants of the entrepreneurship support is primary derived from self-reported impacts and MI.

Overview

The Business Wales programme has become successfully embedded as the main business support service in Wales. It is widely known as a one-stop shop in which solutions can be identified or support brokered. The brand is recognised and trusted, and the majority of participants and stakeholders view the service as straightforward to access and simple to understand. 

Partnerships, collaborations, and referral activities have strengthened as the programme has progressed and stakeholders reflected on how this had enhanced outreach provision as well as engagement with local communities and underrepresented groups. The review of Welsh Government-funded Enterprise Hubs^{[footnote 3]} echoed this perspective in identifying the important role that the Hubs play in engaging entrepreneurs who would not have necessarily engaged with mainstream Business Wales provision. Furthermore, linkages to provision for growth-oriented businesses have also been strengthened (including that associated with the DBW and SMART provision). Several stakeholders, however, reported how levels of integration could be strengthened further. 

Recommendations

• Management information gathered through future programmes of support should capture the prevalence of referrals between service provisions. This will provide useful insight into the extent of integration between various programmes of support.
• That suppliers be performance-managed to encourage appropriate referrals between service provisions.

Context

A host of socioeconomic data illustrated the importance of continued investment in business and entrepreneurial support services, given the shortfall in rates of productivity and entrepreneurial activity in Wales relative to other home nations. For SFBE, the support on offer represented the logical next step in service provision following the investment in superfast broadband infrastructure as part of the previous round of EU Structural Funds. 

Over time, the socioeconomic context changed markedly, which influenced performance. External shocks associated with the COVID-19 pandemic, Brexit, and the cost-of-living crisis being the more obvious influences. Prior to the pandemic, however, the strengthening labour market in Wales and the low rates of unemployment led to a reduction in the demand for necessity-based entrepreneurial activity. In turn, this, alongside the introduction of the New Enterprise Allowance, is thought to have led to lower-than-expected demand for Entrepreneurial Support. 

Progress

The operations were designed and launched as strategic backbone projects within the ERDF programme and, therefore, benefitted from early investment and commencement. Furthermore, their designation as backbone projects likely helped their embeddedness in the business support landscape. A challenge of being one of the early schemes in a round of funding via the EU Structural Funds is the heightened risk that there may remain some gaps in knowledge or information, or some elements on evidence requirements that may lack clarity. This may have influenced some of the early issues encountered in the programme regarding the capture of evidence that conformed to EU requirements.  

Recommendation 

• That an early review of data capture processes against eligibility criteria be undertaken during the implementation phase of an operation. 

Service delivery models for L1–4 Core & Growth across the Entrepreneurship Support and SME Support operations are distinct from that offered through the BW AGP. L1–4 Core & Growth has required active and sustained promotion, working closely with the Welsh Government and building a greater level of local community collaboration and engagement over time. SFBE has also relied on the Business Wales brand to boost levels of engagement, whilst the BW AGP has relied more heavily on Relationship Managers to identify suitable businesses for the programme.

The provision has secured nationwide coverage; however, it is evident that in the BW AGP there is a tendency for rural or geographically peripheral locations to be underrepresented. This underrepresentation may, however, be symptomatic of the nature (in terms of both sector and growth orientation) of businesses operating in those locations. Moreover, the timing of enrolment in each programme varied. Three quarters of BW AGP clients were enrolled in the programme by the end of 2018, whereas only 30% of SFBE clients had enrolled in the programme by that point. 

The Entrepreneurship Support and SME Support operations were short of their profile against several of the indicators in the lead-up to the mid-term evaluation. The operations did, however, illustrate strong performance in relation to employment growth and export sales and largely sustained that performance throughout the remainder of the operations (despite the issues surrounding the pandemic). 

Following the mid-term evaluation in 2018, the programme gained momentum through to March 2020 (when the COVID-19 pandemic reached the UK). At this point the service rapidly and successfully pivoted towards a remote service offer, geared towards resilience (rather than growth). That shift in service offer to remote provision supplemented an adjustment in the nature of support delivered, which at times required a much greater emphasis on pastoral and well-being support. 

Remote, online provision is now widely accepted by participant businesses and is a more efficient means of service delivery. However, in-person delivery can play a key role in addressing certain issues and helping to attract underrepresented groups to support. It is likely, therefore, that a blended service offer will be needed in future provision. 

Recommendation 

• That a blended service model be retained in future support programmes due to its value in providing additional flexibility and efficiencies in service delivery whilst also retaining the ability of in-person support where appropriate. 

Against the background of a pre-existing shortfall against their profile, the pandemic slowed progression for the Entrepreneurship Support and SME Support operations against the majority of indicators, further influencing the disparities in performance against profiled targets.  

Stakeholders referred to the initial targets as challenging, with some describing them as overambitious. It is understood that historical performance had influenced the setting of target indicators, but with future programmes based around a similar (albeit blended) delivery model, contextualising targets against performance in the latter stages of this programme will be important.

Recommendation 

• In future programmes in which consistent performance indicators are applied, progress in their delivery within the latter stages of this programme should be used to inform the setting of targets. 

The early performance against Cross-cutting Theme (CCT) indicators was poor and at the point of the mid-term evaluation the programme was operating at a fraction of the anticipated profile against these indicators. Following the pandemic, however, there was a considerable upturn in performance against CCT indicators. This is reflective of targeted activity to embed this more centrally within service provision, which has led to almost two thirds of all participant enterprises contributing to CCT target indicators in 2022 (compared to less than one fifth of participant enterprises in 2016). 

Recommendation 

• Future support should retain and build upon good practice associated with effective integration of provision that contributed to CCT indicators. The loss of EU funding and, therefore, the requirements for contribution to CCTs should not lead to deviation from this activity. 

The shifting more generally of service provision so as to fulfil targets has not been without concern, with some stakeholders identifying the tension that this may create between delivering target indicators (particularly those indicators associated with hours of support) and delivering support that is reflective of client needs. 

Recommendation 

• That less emphasis be placed on process-oriented indicators (such as reaching a threshold in the duration of support delivered) in future programmes of delivery. 

Perceptions of support 

The participant survey illustrated relatively high rates of satisfaction amongst respondent businesses, with the journey towards provision perceived to be particularly straightforward. This simplistic model from the perspective of the customer was a key objective for Business Wales (with an emphasis on hiding the complexity behind the various contracts) that appears to have been successfully achieved. 

A polarised perspective on support was evident, however, amongst some participants. The majority of surveyed participants described their expectations as having been exceeded. However, almost one fifth of those who received SME Support described being dissatisfied with the way in which the support was provided. An analysis of the duration of support suggests a correlation between the amount of support provided and the level of satisfaction with that support. Whilst it is likely that clients who were unsatisfied with the service were more likely to have withdrawn for the support early, concerns regarding a lack of follow-up communication with clients after receiving support was a primary influence on participants being unsatisfied. 

Recommendation 

• Future programmes need to provide clarity of message regarding the nature of support available whilst also managing expectations amongst the client group regarding the duration of support that they might receive. 

Outcomes and Impact: L1–4 Core & Growth Programme and BW AGP

The L1–4 Core & Growth and the BW AGP has generated good levels of job creation, both within the recorded target indicators and the CIE. According to BAS data, 70% of participant enterprises in L1–4 Core & Growth or the BW AGP have created employment since receiving support. The programme also represents good value for money on a cost-effectiveness basis when calculating the cost per job created amongst supported businesses. Self-reported attribution of jobs through the survey, however, is possibly lower than anticipated. 

The FTE salaries of jobs created have decreased amongst those businesses enrolled in the latter years of the programme. Whilst this is likely to be an influence of the profile of businesses engaging with support (with participant enterprises of 0 to 1 employees becoming more prevalent in the programme each year), the pattern remains when the size of the business is accounted for. Employment brings about a range of economic and well-being benefits. In future research it would be useful to gather additional insight on the jobs created including the hours and contractual security of those roles and whether those positions have been taken up by individuals who were previously unemployed. It would help provide further clarity on the added value of those jobs created against a pattern of falling salaries that on face value suggests a reduction in productivity per job created and in the quality of those jobs.

Over half of respondents in receipt of Entrepreneurship Support started a business following the receipt of that support. Without the support, 16% would not have set up a business, whilst a further 3% were unsure as to whether they would have done so. 

Around half of respondents to the survey described increases in productivity, sales and profitability as a direct result of the support. Furthermore, almost one third of those participating in Entrepreneurship Support describe the support as having made a vital contribution towards the performance of their business. These are strong outcome indicators illustrating value in the support not picked up through ERDF indicators.

Whilst sales growth again appeared to be strong for SME Support provision, indicators of productivity relative to the comparison group were less strong (albeit with a catch-up effect evident three years post-support). The lack of emphasis in terms of target indicators for domestic sales growth or productivity may have influenced this outcome.

Outcomes and Impact: SFBE

The key outcome indicator for SFBE operation is the number of enterprises supported to introduce new-to-the firm products. The operation surpassed target against this indicator and delivered them more cost-effectively than the target set out within the original business plan. Furthermore, on average each business supported by SFBE adopted 2.3 new-to-the-firm products illustrating the success of the operation against this target indicator. 

Analysis of the nature of the new-to-the-firm products introduced shows that almost one fifth (19%) were considered high technology products whilst a further 16% were considered medium technology products. More generally, for business across the majority of industrial sectors support, over half of the new products introduced were a high or medium technology solution^{[footnote 4]}.  

The CIE suggests little effect in terms of employment growth for those in receipt of SFBE support. This is likely to relate to the fact that the SFBE provision was never intended to deliver job outcomes. The primary outcomes expected for this support related to productivity and efficiency gains, indeed 85% of survey respondents felt the support made at least some contribution to their business performance whilst 59% report productivity increases in their business. Furthermore, the associated emphasis of SFBE support on the introduction of new to the firm products would likely experience a time lag between their introduction and any tangible changes in sales and employment that would be captured in the CIE. 

Recommendations

• A broader range of target outcome/result indicators associated with business support initiative should be made available. This would enable operations (initiatives) to target and capture those outcome indicators most likely to be affected by the support on offer.
• Consider a greater emphasis on sales generation and productivity gains amongst the suite of target indicators for future, growth orientated, business support provision. 

Future Provision

The pandemic continues to influence business performance, with over half of survey respondents reporting that it affected their margins to a great extent in their last financial year. 

Recommendation

• That future support provision continues to focus on activity that includes elements supporting the business to recover from, and adjust to, the changing economy arising from the COVID-19 pandemic.

Stakeholders widely welcomed further funding for BW. They see it as a critical element of the business support landscape in Wales and as a trusted and known source of impartial advice in the business community. Its flexibility in responding to business needs over a turbulent socioeconomic period was widely praised, but there are concerns as to whether that ability to pivot and respond to future crises would remain without additional resources. 

The UKSPF is viewed by most stakeholders consulted as part of this evaluation with concern. BW has simplified the business support landscape, yet stakeholders widely believed that the UKSPF may lead to an increasingly complex, fragmented and inconsistent business support offer. 

Recommendation

• Extensive work with local authorities and communities to foster strong relationships and share intelligence will be necessary to ensure successful alignment of Business Wales provision with that which emerges through the UKSPF.

[1] Zaidi, A; Shah, P; and Beadle, S (2018). Evaluation of Business Support Services in Wales: a report for the Welsh Government. Cardiff: Welsh Government.

[2] Oliver Allies, Paula Gallagher, Michelle Hollier, Michael Pang, Benjamin Reinertz, Prabhat Vaze, and Paul Woodcraft.; (2023). ERDF Support for Business Evaluation. Cardiff: Welsh Government, GSR report number 87/2023.

[3] OB3 (2022) Review of Welsh Government Entrepreneurial Support Services Enterprise Hubs, Welsh Government. 

[4] High technology products include cloud technologies, customer relationship management, IT Security/Anti-Malware, Specialist Hardware or Software (bespoke) and Unified communications. Medium technology includes online booking solutions, document management/file sharing software, electronic/card payment, server infrastructure.

Report authors: Oliver Allies, Michael Pang, Benjamin Reinertz and Prabhat Vaze

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Research, Monitoring and Evaluation team
Knowledge and Analytical Services
Welsh Government
Cardiff
CF10 3NQ
Email: research.evaluation@gov.wales 

Social research number: 3/2025
Digital ISBN 978-1-83715-160-8