Formative evaluation of Jobs Growth Wales+: final report (summary)
Research exploring the process, uptake, and outcomes of the Jobs Growth Wales+ programme in years 1 (2022/23) and 2 (2023/24).
This file may not be fully accessible.
In this page
Programme overview
Jobs Growth Wales+ (JGW+) launched in April 2022 (initially to run to 2026) aiming to deliver individualised training, development, and employability support to 16 to 18-year-olds [footnote 1] who are assessed as NEET (Not in Education, Employment or Training) on joining the programme.
Initially operating with a referral system via Working Wales only, the delivery model was adapted to also allow for referrals from Contractors and Engagement and Progression Coordinators (EPCs). This was done to improve access for young people who are the ‘hardest to reach’.
The nature and scale of the participants’ needs and barriers, and their distance from education or employment, inform to which strand of the JGW+ programme the young person is referred (engagement, advancement, or employment). The JGW+ services are delivered by five Prime Contractors across four regional areas of Wales.
Methodological approach
In July 2022, Wavehill were commissioned to undertake a formative evaluation of JGW+, to:
- track and record programme engagement, enrolment, and participation
- undertake a process and annual outcomes assessment of the programme
- establish an impact evaluation framework for the programme
This final phase of the evaluation provides analysis of the progress and success of JGW+ since its launch, building on findings from the scoping and interim phases.
The evaluation has involved:
- analysis of programme management information, including the Lifelong Learning Wales Record (LLWR) dataset and Contractor Monitoring Information of participants, covering the period April 2022 to July 2024
- stakeholder interviews (virtual) with Welsh Government staff (n=10); Contractors and Sub-Contractors staff involved in the management and delivery of the programme (n=31); and local authority representatives, typically the Engagement and Progression Coordinator role (n=29) from 17 (of the 22) local authority areas in Wales.
- participant fieldwork (through a telephone survey) with two groups from four of the five Prime Contractors:
- a ‘core’ survey with existing participants and recent completers/’exiters’ enrolled between 1st October 2023 and 31st July 2024 and remained on the programme until at least May 1st, 2024 (n=158)
- a ‘post six-months’ survey with participants enrolled between 1st October 2023 and 1st April 2024 and who had exited the programme over six months prior to survey engagement (exiting prior to May 1st, 2024) (n=62)
Methodological limitations
Whilst the distribution of survey responses largely mirrored targets, those responding were less likely to have secured positive outcomes than reported in the national performance data. This may lead to a negative bias in feedback, particularly around additionality of the support.
One contractor was not represented in survey responses leaving a lower representation of feedback from those residing in the north east of Wales.
Not all local authorities have engaged in the evaluation, with some areas not engaging at either interim or final evaluation phase. There are therefore no local authority perspectives from Bridgend, Cardiff and Powys.
Main findings and conclusions
Programme design and context
Whilst JGW+ launched in 2022, it was designed in 2019 to 2020 and crucially, prior to the COVID-19 pandemic. A series of in-programme adaptations were made to programme design and delivery, primarily in response to young people presenting more severe and complex needs than expected at the point of design. The willingness to adjust the delivery model reflected what Contractors described as a shift in approach from Welsh Government when compared to previous programmes, towards a partnership model in which Contractors work together, and a ‘can do’ attitude to addressing identified programme challenges.
Those adaptations have been universally welcomed by Contractors and external stakeholders. They illustrate the importance and value of post-implementation adjustments to a policy intervention to reflect changing circumstance and emerging evidence.
The model itself and the guidance around it are, however, relatively complex and there remain instances where Contractor (or Subcontractor) representatives are unaware of, or have misunderstood the latest guidance. There are also concerns amongst EPCs that little information is shared about participants once they have enrolled (and particularly whether an individual has secured an outcome on exit from the programme).
Recommendation 1: Clear communication regarding adjustments to the service offer is necessary from the Welsh Government to Contractors, to be disseminated to all frontline staff, Sub-Contractors, and EPCs to ensure that consistency of the offer is retained.
Recommendation 2: Further engagement with EPCs should be undertaken to explore the extent to which they are receiving communication in relation to the achievements and outcomes of JGW+ participants from their local area.
Engagement
The JGW+ programme benefits from a good level of existing awareness amongst the target group with levels of demand for support outstripping the level of resource available. Between April 2022 and July 2024 there were 11,972 programme starts, and 10,008 unique participants enrolled on JGW+.
Whilst available eligible population data is of limited quality, evidence suggests that 30 to 40% of the eligible cohort in Wales may be engaging with JGW+. Furthermore, one quarter of the participants on the programme are presenting with additional needs. The evidence therefore indicates a service offer that may be better suited to engaging and enrolling the target group of young people than previous policy interventions.
Word of mouth and promotional outreach activities within secondary schools have emerged as effective in raising awareness of the JGW+ programme. Several factors are likely to have influenced engagement rates including the quality and range of support on offer and the rates of compensation (training allowances, free meals and transport for participants and the induction fee for Contractors) that were secured.
Survey respondents highlighted that whilst the financial support (through the higher rate of training allowance) was a factor, it was not the main motivational factor in engaging with the programme. Contractors meanwhile felt that the increased training allowance aided participant retention on the programme.
The levels of demand for the programme have led to Contractors having to carefully manage their annual profile of enrolments to avoid running out of funding. Waiting lists have at times been established to help manage volumes, particularly in urban areas. Contractors suggested that this considerably heightens the risk of potential participants disengaging.
The management of annualised budgets by Contractors is further complicated by considerable fluctuations in demand for the programme during a calendar year. There are consistent spikes in enrolment each July, and to a lesser extent, each October. Those in July appear to be associated with a transition out of secondary education to JGW+ and for many, they are NEET for a very short time or not at all, with some reporting enrolment to JGW+ whilst still in school.
The programme appears effective at engaging during the summer months however the additionality of support to this specific cohort (when compared to enrolments at other points during the year) is less clear. It should also be acknowledged that the process of transitioning from secondary school at 16 into some other form of EET is beset by risk of disengagement and JGW+ (and specifically the pre-engagement offer through ‘Get Ready’) provides a useful mechanism for retaining some form of engagement over the summer period.
Whilst the programme enjoys a good level of awareness amongst young people, there is a high degree of geographical variability in the likelihood of an individual engaging on the programme. For example, a young person aged 16 to19 in Torfaen is eight times more likely to enrol on JGW+ than one in the more rural locations of Powys, Gwynedd or Ceredigion.
There is some evidence that the general population of NEET young people in Wales varies by geography, however where there is variation, there is limited correlation with rates of engagement on JGW+. This suggests other factors are influencing the propensity to engage in the support. Such factors may include, levels of deprivation, issues of rurality and challenges of accessing a physical location for JGW+ advice and guidance. Contractors also identified the need for flexibility when young carers engage with the programme and physical attendance can sometime be difficult.
Recommendation 3: Review access to support, including the level and the extent to which individuals can engage online, to support achieving equitable access to JGW+ across Wales.
Prospective JGW+ participants are most commonly directly referred to the programme, either through the Contractor or EPCs. The two-pronged approach however to referral and engagement (via either Working Wales or through direct referral) appears to be working well, enabling engagement routes to be largely learner-led. The Assessment and Referral Report (ARR) form provides a useful mechanism for capturing insight about a participant, where that information is provided in a comprehensive manner. Restrictions around sharing school related data including Individual Learning Plans (ILPs) often leads to delays in understanding participant needs until they choose to share this information (typically after trust is established with staff member), or until it can be identified through a needs assessment. In this regard the importance of the conversational approach as part of the initial engagement of young people cannot be understated. Building a rapport and a sense of trust at an early stage in their engagement is critical to understanding participant needs, sustaining their engagement and securing positive outcomes.
A challenge identified as part of the enrolment process is the managing of participant expectations. Young people are sometimes enrolling with unrealistic expectations of a career that does not reflect their existing qualifications, or are unfamiliar with the challenges they may have faced had they looked to secure employment. This is not typically addressed in the process of referral and is a factor that is typically looked at post engagement.
The nature of challenges faced by participants means that over three quarters of participants are enrolled onto the engagement strand of the support. For some participants, it is felt that this is too advanced with a view that more widespread usage of the (or another) pre-engagement offer (currently ‘Get Ready’) is necessary.
Patterns of delivery of Get Ready as an offer appear influenced by available resources, provider interest and seasonality, with limited evidence to suggest that its current delivery is influenced by learner needs. Of those residing in Neath Port Talbot and Flintshire, almost a quarter of participants engaged in the Get Ready offer, whereas none of the participants residing in Ceredigion have accessed Get Ready. It is believed that available funding is likely to be a factor in the variation in its usage, whilst it is unclear if there are other factors that are influencing geographical variation.
Recommendation 4: Explore with Contractors what factors are influencing the extent of their usage of the pre-engagement ‘Get Ready’ strand of support to support more equal access across the programme.
Reportedly, the number and severity of complexities and barriers faced by participants continue to increase. This is leading to increased prevalence of challenging (disruptive and sometimes violent) behaviour and safeguarding issues for Contractors to manage.
Support provision
The service offer through the mainstream elements of the programme (primarily the engagement and advancement strands) is seen to be flexible, responsive and learner led. There is widespread praise for the nature of support offered through the programme which has been refined over time.
The programme and its related support are widely considered as accessible and impactful, particularly in relation to addressing mental health and wellbeing and in supporting young people in the development of ‘soft skills’. The data capture and monitoring requirements for the programme however are largely shaped by the LLWR, which fails to sufficiently capture the severity and complexity of barriers nor any insight into the personal wellbeing of a participant. Analysis of ILPs (where this information is available) for example shows nearly one quarter of records evidencing multiple support needs for each participant with associated narrative illustrating the complex situations that some participants face.
Recommendation 5: Monitoring evidence that complements the requirements of the LLWR should be gathered as part of the engagement process with a young person (once trust is established) and could form part of the ILP. This should include capturing wellbeing indicators in a quantitative or categorised manner to allow them to be monitored and analysed.
Recommendation 6: A consistent approach should be established across the programme for the capture of soft outcomes and distance travelled. To ensure this has the support of Contractors it is recommended that they, collaboratively lead on the system’s design and development.
Recommendation 7: A consistent approach should be adopted across all Contractors for the recording of key information held in the ILP. This would enable a more comprehensive qualitative analysis of learner barriers, aspirations and activities.
Support provision delivered through JGW+ has slowly, steadily shifted towards wider life skills and basic skills and away from provision typically associated with individuals close to employment. This is likely to be a reflection on the circumstances of participants who engage. Moreover, the employment strand of the programme is little used with only 1% of programme starts on that strand. Contractors reported that many of the young people supported faced additional barriers and situations where typically they were not ready for work.
More generally regarding employer engagement, Contractors flagged a limited willingness from employers to take on young people given the supervisory costs involved, a concern that participants seemed immature or non-committal and a more general lack of these opportunities in certain locations.
Participants referenced delays in securing placements whilst Contractors highlighted that the allowance offered to participants was lower on placement meaning there was greater financial gain in remaining in the learning facility.
Recommendation 8: Investigate the disparities in provision of allowance on placement compared to in learning facilities, and the impact this has on motivations for completing placements.
Despite these challenges around exposure to employment opportunities, management information shows that across Wales, over one fifth (21.6%; 1785/8270) of unique individuals secured either part- or full-time employment on exit from JGW+. Again, there is geographical variation in these destinations with over one quarter of participants in Pembrokeshire, Rhondda Cynon Taf, Swansea and Torfaen securing employment compared to less than 10% of those in Cardiff.
Moreover amongst the six-month post participant group there is an increase in prevalence of positive destinations compared to point of exit with those in employment typically transitioning to more secure contracts or a greater number of hours per week.
Exposure to employment opportunities more generally (primarily through work placements) is a further area of geographical variability with, JGW+ participants in Ceredigion and Carmarthenshire being almost 20 times more likely to undertake some form of work-based hours as part of their engagement than those residing in Flintshire.
Recommendation 9: Engage Contractors to explore factors surrounding the variation in participant journeys on the JGW+ programme to identify what support can be offered to aid the provision of a more universally consistent offer.
Outcomes and impacts of support
Positive outcome rates on the programme have steadily improved over time with the programme surpassing targets against the number of programme starts. Within published data however there is little distinction between progression outcomes (where an individual transitions from JGW+ strand to another) and final outcomes (their outcome at point of exit from the programme). Whilst positive movement between programme strands is an important milestone to acknowledge, this may be inadvertently misleading perceptions on scale of engagement and destinations upon exit from the programme.
Recommendation 10: Capture positive movements between strands, and final outcomes for unique individuals as separate indicators in the reporting of programme performance.
Considerable geographical variation is further observed in the prevalence of positive outcomes amongst participants, with those in Powys and Pembrokeshire three times more likely to secure a positive outcome than those in Torfaen. Similarly, those residing in Cardiff are almost three times more likely to leave with a negative outcome than those residing in Wrexham and Flintshire.
The programme has been instrumental in helping learners overcome barriers, develop essential soft and employment skills. JGW+ participants further credited the programme with providing valuable clarity and guidance on their career options.
Outside of employability, the programme had a substantial effect on learners' wellbeing and mental health and leaners particularly commended the guidance of tutors and staff for their supportive and understanding approach.
Learners highlighted increased confidence and enhanced overall wellbeing as the most transformative outcomes of their experience. They also frequently reported improvements in their ability to manage personal challenges, such as social anxiety. For those who faced mental health challenges before joining JGW+, these improvements underscored the importance of addressing personal development and wellbeing as foundational elements before pursuing employment-specific goals.
Recommendation 11: That a comprehensive social cost–benefit analysis of JGW+ be undertaken to fully capture both the societal and the economic costs and benefits arising from the programme in order to help assess the programme’s return on investment.
The programme’s ability to adapt and respond to participant needs is illustrated by the relatively limited variation in positive outcomes achieved by characteristic. In a similar pattern to that found in the interim evaluation, those with some form of Welsh language capability are statistically more likely to secure a positive outcome than those without.
The scale of the programme and the extent to which it appears to be engaging with the eligible cohort means it is possible that JGW+ is having an influence on socio-economic indicators of performance in Wales. Recent data on unemployment rates amongst the 16 to19 year old cohort illustrates a marked fall and emerging divergence (where comparative data is available) from trends in England. Further exploration of this data (as part of a summative evaluation) is necessary. Furthermore, details on economically inactive numbers are required alongside those who are unemployed to more clearly understand the effect the programme may be having.
Amongst year 11 leavers, impact or added value is unclear with rates of NEET or unknowns remaining static following increases from 2021 to 2022. These indications alongside various other trends as highlighted earlier within this section suggest lower levels of additionality arising from those enrolled on JGW+ in July (specifically those transitioning out of Year 11) than those enrolling on JGW+ in other months of the year.
Recommendation 12: That further exploration of the specific characteristics and situations of those enrolling on the programme in July of each year be undertaken with Contractors and EPCs.
Recommendation 13: That future fieldwork with participants of JGW+ engage with a cohort who have enrolled on the programme across the calendar year to mitigate any potential influence of monthly trends.
Footnotes
[1] The age group for eligibility to JGW+ was extended to 19 in January 2023.
Contact details
Report authors: Jakob Abekhon, Oliver Allies, Paula Gallagher, Katy Simms, and Sarah Usher
Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.
For further information please contact:
Eloise Gowing
Social Research and Information Division
Knowledge and Analytical Services
Welsh Government
Cathays Park
Cardiff
CF10 3NQ
Email: KASEmployabilityandSkillsResearch@gov.wales
Social research number: 54/2025
Digital ISBN: 978-1-83715-848-5