Analysis of Flying Start outcomes using linked data: emerging findings

This evidence brief provides the first outcomes of research based on linking datasets with those families in receipt of Flying Start services. 

This evidence brief only relates to the City and County of Swansea Council. Further analysis combining data for additional local authorities will be published in future.

Health outcomes for 2009-17 and education outcomes for 2011/16 were compared for children living in the Swansea area who had received Flying Start services (the ‘Flying Start group’) and children who had not received Flying Start services (the ‘Non-Flying Start group’).

These emerging findings suggest:

• A limited, preliminary indication of a possible positive impact of Flying Start on low birth weight and births to teenaged mothers. However, the finding is based on combined figures for births that would have brought mothers into contact with Flying Start and births to mothers already receiving Flying Start. Further work would be needed to focus only on births to mothers already receiving Flying Start in order to identify an association with Flying Start. 
• No impact on overall A&E attendances. However further analyses will be carried out in future releases looking at types of A&E attendances. 
• A possible ‘protective effect’ of Flying Start on hospital admissions. Hospital admissions increased between 2014-15 and 2016-17 for both the Flying Start group and the Non-Flying Start group but the increase was smaller in the Flying Start group.
• A possible positive impact of Flying Start on primary school absences. Primary school absences decreased between 2011/12 and 2015/16 for both the Flying Start group and the Non-Flying Start group but the decrease was greater in the Flying Start group.
• A possible positive impact of Flying Start on unauthorised primary school absences. Looking at the period since fixed penalties were introduced, i.e. 2013/14 to 2015/16, unauthorised absences decreased for both the Flying Start group and the Non-Flying Start group but the decrease was greater in the Flying Start group. 

For these emerging findings, it has not been possible to complete the in-depth kinds of analysis necessary to show a margin of error for the results or to examine any effects in more detail. More in depth analysis will be published in future evidence briefs.
 

ADR Wales is a new innovative partnership. It brings together data science experts at Swansea University Medical School, staff from the Wales Institute of Social and Economic Research, Data and Methods (WISERD) at Cardiff University and specialist teams within the Welsh Government. Together they develop new evidence which supports the Welsh Government’s national strategy, Prosperity for All.

ADR Wales uses the SAIL Databank at Swansea University, to link and analyse anonymised data. This process enables the Welsh Government to understand more about the relationship between different areas of public service delivery and gain a better understanding of people’s experience as they move through different services. This supports the development of collaborative and integrated policy to improve the lives of people in Wales.

Flying Start aims to improve the life chances of young children under the age of 4 living in some of the most disadvantaged areas in Wales. This evidence briefing presents initial findings about the health and educational outcomes of those in contact with Flying Start services in the Swansea Local Authority area.

The core elements of Flying Start are:

• an enhanced health visiting service (where the health visitor caseload is capped at 110 children)
• access to parenting support
• access to speech, language and communication support (previously described as Early Language Development) 
• funded part-time high quality childcare for 2-3 year olds.

These services are universally available to all children aged under 4 years and their families in the areas in which the programme runs. In addition to these elements, local authorities are able to apply a degree of flexibility within the Flying Start programme by offering support through Outreach. Outreach enables a small number of families living outside Flying Start areas to access the support they need based on agreed criteria. Using local knowledge and an assessment of priority, local authorities can aim to ensure those most in need receive this service.

Official statistics for Flying Start are published on the Welsh Government website.
 

Historically, analysis of Flying Start tended to highlight some positive effects of the programme, but these either were very weak effects or were more qualitative in nature and based on the views of parents.

Two more recent research projects attempted to build on these by using existing administrative data to look at outcomes for children who were living in Flying Start areas and therefore eligible for (but not necessarily in receipt of) the programme. The Flying Start Data Linking Demonstration Project published in January 2014 analysed administrative data for children resident in Flying Start eligible postcodes to assess health outcomes. In addition, the WISERD Project: Flying Start evaluation: educational outcomes published in January 2017 explored education outcomes for Flying Start Eligible Children. Due to the way data has been collected in the past these projects were unable to examine outcomes by the level of engagement with the programme.

The aim of the Flying Start Individual Level Data Collection Pilot project is to collect information about Flying Start children including their level of engagement with the programme. The intention is to monitor and evaluate the progress of Flying Start children during and after their engagement with the programme in order to explore their health, education and broader outcomes.

For this purpose, anonymised individual-level data for those who have engaged with Flying Start services is being provided to the SAIL (Secure Anonymised Information Linkage) Databank directly by local authorities.

Once in SAIL, the anonymised data is linked with other datasets already held in SAIL in order to:

• analyse delivery and take-up of Flying Start services
• investigate current and future outcomes for children receiving and not receiving Flying Start services
• explore differences in outcomes for different patterns/levels of engagement with Flying Start services.

The analysis reported in this article has used a data linkage approach for a single local authority in Wales, the City and County of Swansea Council. These are the first results produced using this approach. Further analysis combining data for additional local authorities will be published in future. 

Anonymised individual-level Flying Start data for the City and County of Swansea Council was linked to data from the following datasets:

• Emergency Department Dataset Wales (EDDS) around Accident and Emergency (A&E) attendances 
• Patient Episode Dataset for Wales (PEDW) around hospital admissions, 
• Welsh Demographics Service (WDS)
• National Community Child Health Dataset (NCCHD)
• National Pupil Dataset (NPD) and Pupil Level Annual School Census (PLASC) for attendance records

For further information on data matching and linking please see Data Matching and Linking.

A comparison group of children in Swansea who did not appear in the Flying Start data was also selected. The health and education outcomes for the two groups, i.e. those resident in Swansea who appear in the Flying Start data (‘Flying Start group’) and those who do not appear in the Flying Start data (‘Non-Flying Start group’), were compared.

It should be noted that, for this Emerging findings report, it has not been possible to complete the in depth kinds of analysis necessary to show a margin of error for the results, nor to examine any effects in more detail. More in depth statistical analysis will be published in future reports.

Outcomes were analysed for children either born or who moved into the City and County of Swansea before the age of 5 years. It should be noted that for those who moved into a Flying Start area, the analysis may include events that occurred before their Flying Start referral date, potentially leading to an under- or over-estimate of the effects reported in the 'Emerging findings' section.

Statistics for a number of indicators have been produced by summarising the linked data for both the Flying Start and Non-Flying Start groups. A common method is used for each group and dataset. Where possible, figures for Wales have been extracted using the same method. The range of time periods for which data are available vary between datasets. As a result statistics have been produced for time periods up to the latest possible available (i.e. 2016-17 or 2015-16).

When considering this analysis, it should not be assumed that any trends presented are solely due to the Flying Start programme as the results are subject to a range of factors including other government policies, socio-economic characteristics and behavioural and attitudinal drivers.

The number of children in the Swansea Flying Start group is 8,071 and the overall number of children in the Swansea Non-Flying Start group is 28,722. Some of the patterns shown in the graphs below may be partially explained by small numbers which can occur when data are broken down by year below local authority level.

The age, gender and socio-economic composition of the Flying Start and Non-Flying Start groups changed over the time period covered by the analysis. This is because children were moving in and out of both groups due to births, deaths, moves in and out of the Swansea area and children ‘ageing out’ of the programme at the age of 4 years. The age composition also differed between the Flying Start and Non-Flying Start groups. For example, there was a higher proportion of children aged 0-1 years in 2009-10 for the Flying Start group compared with the Non-Flying Start group.

As expected, both groups contained a slightly higher proportion of boys than girls in both groups (51% and 49% for Flying Start boys and girls respectively, compared with 52% and 48% for Non-Flying Start).

Why are more boys than girls born every single year?

In addition, and as would be expected given the targeting of the Flying Start programme, 91% of the Flying Start group resided in the most deprived areas in Wales (quintile 1 or 2 of the 2014 Welsh Index of Multiple Deprivation).

As noted above, for this Emerging findings report, it has not been possible to complete the in depth kinds of analysis necessary to show a margin of error for the results, nor to examine any effects in more detail. More in depth statistical analysis will be published in future reports.

This section presents headline findings for children aged 0 to 4 years in the Flying Start and Non-Flying Start groups for:

• low birth weight
• births to teenaged mothers
• overall attendances at Accident & Emergency departments (all reasons) 
• overall hospital admissions (all reasons) 
• primary school attendance and absence for Years 1 and 2

Low birth weight and births to teenage mothers

Factors like low birth weight and teen pregnancy are known to be related to deprivation and poor outcomes. Low birth weight (i.e. the percentage of live single births with a birth weight of under 2.5kg) is included in the Welsh Index of Multiple Deprivation (WIMD) and elsewhere as a proxy indicator for deprivation.

Figure 1 shows the incidence of low birth weight for Wales and for the Flying Start and Non-Flying Start groups. Figure 2 shows the relative incidence of births to teenage mothers since the introduction of the Flying Start programme for Wales and for the Flying Start and Non Flying Start groups.

Figures 1 and 2 are primarily included to give an indication of the differences between the Flying Start and Non-Flying Start groups. However, for these Emerging findings it was not possible to separate births that would have brought mothers into contact with Flying Start from births to mothers already receiving Flying Start. For the latter, it could be argued Flying Start could have affected issues like low birth weight and teen pregnancy. As noted above, some of the volatility evident in Figure 1 and possibly Figure 2 may be explained by small numbers. 

Overall, Figures 1 and 2 both show what appears to be a downward trend for both the Flying Start and Non-Flying Start groups, i.e. in both low birth weight and births to teenaged mothers. However, possibly for low birth weight and certainly for births to teenaged mothers, the decrease between 2009-10 and 2015-16 appears greater in the Flying Start group than in the Non-Flying Start group. This may provide a limited, preliminary indication of a possible impact of Flying Start on low birth weight and births to teenaged mothers. Further work would be needed to focus only on births to mothers already receiving Flying Start in order to confirm an association with Flying Start. Future reports will seek to separate these two groups to explore the issue further. This work would only be possible once data is available for more than one local authority, since numbers would likely to be too small for Swansea.

Accident and Emergency (A&E) Attendances

Figure 3 shows the overall rates of attendance at Accident & Emergency departments in Wales for children aged 0 to 4 years in the Flying Start and Non-Flying Start groups by financial year, calculated per 100 children. It should be noted that these results include minor injury units from 2011-12 onwards, results prior to 2011-12 exclude minor injury units. This may have contributed to the increase shown between 2010-11 and 2011-12.

The Flying Start group had higher rates of attendance than the Non-Flying Start group across the whole period. There are a number of possible explanations for this, for example proximity to A&E departments and variations in incidence of injuries and poisonings. This may indicate a higher level of need in the Flying Start group. Overall, the Flying Start group follows a similar trend to the Non-Flying Start group. Future publications will investigate this further.

Hospital admissions (PEDW)

Figure 4 shows the rate of hospital admissions in children aged 0-4 years in the Flying Start and Non-Flying Start groups, by financial year. Rates are shown per 100 children and include both emergency and elective hospital admissions. Future reports will examine emergency and elective hospital admissions separately.

The rate of admissions showed consistent small decreases between 2009-10 and 2014-15 for both the Flying Start group and the Non-Flying Start group. It is noteworthy that while the rate of admissions increased by six admissions per 100 children between 2014-15 and 2015-16 in the Non-Flying Start group (from 22 admissions per 100 children in 2014-15 to 28 admissions per 100 children in 2015-16), the rate of admissions for the Flying Start group increased by one admission per 100 children (from 32 admissions per 100 children in 2014-15 to 33 admissions per 100 children in 2015-16). This may suggest a protective effect of Flying Start.

Further work will be carried out to look into reasons for recent increases in hospital admission rates for children. 

Primary school attendance

The WISERD Project: Flying Start evaluation: educational outcomes showed that once Flying Start was introduced, the attendance of primary school aged children living in Flying Start areas tended to improve. Flying Start services are provided for children aged 0-3, so the following graphs are based on data for those in contact Flying Start services linked to data created once they have started school. Published education statistics are based on school years so the graphs below show rates for school years and for children of statutory school age (i.e. 5 years and over) with the 2011/12 data including School Year 1 data only, due to the implementation of Flying Start. Data for nursery and reception, where recorded, is excluded from the results shown because it is not consistently recorded and there is no statutory requirement to provide that data to the Welsh Government.

Figure 5 shows a possible impact of Flying Start on primary school absences based on the percentage of half-day sessions missed over the school year. Primary school absences decreased between 2011/12 and 2015/16 for both the Flying Start group and the Non-Flying Start group but the decrease was greater in the Flying Start group.

School absence can be broken down into authorised and unauthorised absence, measured here in terms of average half days missed per year. An authorised absence is an absence with permission from a teacher or other authorised representative of the school. This includes instances of absence for which a satisfactory explanation has been provided (e.g. illness, family bereavement or religious observance). An unauthorised absence is an absence without permission from a teacher or other authorised representative of the school. This includes all unexplained or unjustified absences. Figure 6 shows average half day sessions missed due to unauthorised absence.

For both groups there were increases in 2013/14 followed by slight decreases in 2014/15. Some of the change shown for 2013/14 may be related policy changes, including the introduction of fixed penalties for unauthorised absence, which may have affected the way certain types of absence were recorded from that year onwards.

Looking at the period since fixed penalties were introduced, i.e. 2013/14 to 2015/16, unauthorised absences decreased for both the Flying Start group and the Non-Flying Start group but the decrease was greater in the Flying Start group.
 

The next evidence brief for the Flying Start Data Linking Project, planned for publication in Autumn 2019, will report on outcomes by level of engagement with Flying Start e.g. childcare uptake.

Future releases are planned to: 

• analyse data from further local authorities
• examine the extent to which Flying Start has achieved the outcomes as envisaged in the Flying Start programme logic model
• examine the extent to which the receipt of Flying Start services by an eligible mother and child affects the outcomes for the remaining (i.e. older) children in the same household
• examine the extent to which the receipt of Flying Start services by an eligible mother and child affects the outcomes for other adults and (i.e. older) children in the same neighbourhood
• compare breastfeeding levels for the Flying Start and Non-Flying Start groups
• present analysis by WIMD groupings
• present analysis by mother’s highest educational qualification level
• present analysis by household composition
• present analysis by tenure
• present analysis by as many as possible of the protected characteristics under the Equalities Act 2010.
   

When considering this analysis, it should not be assumed that any trends presented are solely due to the Flying Start programme as the results are subject to a range of factors including other government policies, socio-economic characteristics and behavioural and attitudinal drivers.

There are also varying amounts of true exposure time, for example, a child born at the end of the yearly cut-off points will have had less time to experience any potential benefits of Flying Start. As noted above, for children who moved into a Flying Start area, the analysis may include health events that occurred before their Flying Start referral date, potentially leading to an under- or over-estimate of the effects reported above.

As noted above, the age, gender and socio-economic composition of the Flying Start and Non-Flying Start groups changed over the time period covered by the analysis due to births and deaths, moves in and out of the Swansea area and children ‘ageing out’ of the programme. The age composition also differed between the Flying Start and Non-Flying Start groups. In this analysis, although the rates only refer to 0-4 year olds, they are not age-adjusted so the findings likely show mixed effects as each age, e.g. 0-1 year olds, have a different trajectory of health service usage.

This article does not report National Statistics, but the findings may relate to National Statistics outputs.

The following sections provide an outline of more detailed quality information.

Relevance

The analysis presented in this evidence brief relates to a high-profile programme funded by Welsh Government and delivered by local authorities and health boards across Wales. We anticipate that the scope of the usefulness of this publication may be far-reaching. It is anticipated that this publication will be used by Welsh Government, local authorities, other public bodies, and the general public, to gain an understanding of the difference Flying Start services are making in Wales.

Accuracy

Administrative data, whilst useful, is not designed for research purposes and can be prone to issues with inconsistency of recording, missing or incorrect data and transcribing errors when moving from paper-based to electronic recording systems.

This analysis does not constitute National Statistics, but may relate to National Statistics outputs and will nevertheless have been subject to careful consideration and detailed checking before publication. More information about official statistics and National Statistics can be found on the UK Statistics Authority website.

Timeliness and punctuality

This publication has been prepared at the earliest opportunity in order to provide initial findings from Flying Start data contained in the SAIL databank. The aim is to provide initial insights into Flying Start using a data linking approach, and a useful starting point for further research but which is useful in its own right.

Due to regular updates to the data contained in the SAIL Databank, it would be possible, in future, to repeat the analysis to include results for additional years. The analysis shown was carried out using data available as at July 2019.

Accessibility and clarity

This publication has been made available as an online report to maximise impact and efficiency using the Welsh Government open data services.

This evidence brief is based on securely linked, anonymised administrative data held in the SAIL Databank. As a result it is not yet possible to make the data available on StatsWales or elsewhere. However, if you would like to analyse this data in the future; please get in touch with us by emailing ADRUWales@gov.wales.

Comparability and coherence

As a one-off analysis for a single local authority in Wales, comparability with other analysis is limited. We do not intend to update this report regularly. Further analysis will nevertheless be produced using the same approach but providing results for greater numbers of local authorities.

As noted above, and as would be expected due to the targeting of the programme, there are significant differences in socio-demographic characteristics between the Flying Start group and the Non-Flying Start group. For example, the Flying Start group represents a more disadvantaged group.

The Non-Flying Start group is therefore to be viewed, not as a statistically robust control group but purely for the purpose of providing the best available comparison.

As noted above, an explanation as to the strengths and limitations of each of the sources for this analysis, and the coherence between them, will be provided in a forthcoming report.

Well-being of Future Generations

The Well-being of Future Generations Act 2015 is about improving the social, economic, environmental and cultural well-being of Wales. The Act puts in place seven well-being goals for Wales. These are for a more equal, prosperous, resilient, healthier and globally responsible Wales, with cohesive communities and a vibrant culture and thriving Welsh language. Under section (10)(1) of the Act, the Welsh Ministers must (a) publish indicators (“national indicators”) that must be applied for the purpose of measuring progress towards the achievement of the Well-being goals, and (b) lay a copy of the national indicators before the National Assembly. 
Information on the indicators, along with narratives for each of the well-being goals and associated technical information is available in the Well-being of Wales report.

Further information on the Well-being of Future Generations (Wales) Act 2015.

The statistics included in this release could also provide supporting narrative to the national indicators and be used by public services boards in relation to their local well-being assessments and local well-being plans.

Next update

This publication is a one-off release and will not be updated. However, due to regular updates to the data contained in the SAIL Databank, further analysis will be published in future, repeating the analysis to include results for additional years and geographical areas.

ADR Wales is part of the Economic and Social Research Council (part of UK Research and Innovation) funded ADR UK.

Tony Whiffen (Welsh Government) and Laura Herbert (Swansea University), along with other members of the ADR Wales team have worked collaboratively to produce this article. 

Statistician: Tony Whiffen
Telephone: 0300 062 8921
Email: ADRUWales@gov.wales

Media: 0300 025 8099

We welcome any feedback on any aspect of this report. Please send your feedback by email to ADRUWales@gov.wales.