Agile Cymru guidance: output and outcomes indicators

This guidance document provides definitions for output and outcome indictors for the Welsh Government’s Agile Cymru programme. It is intended to support those involved in any stage of developing, managing or evaluating a project funded by Agile Cymru. 

Your application will need to set out which outputs the project will deliver during its operational phase and the outcomes that may follow from that in the short and medium term. Short term outcomes could be captured during the operational phase of the project or shortly after completion. Medium term outcomes could be captured many months after project completion where there is a logical link to the project activity but will probably be within a year of completion. There will need to be a clear intervention logic between the type of activity proposed and the outputs and outcomes proposed. You do not need to include every output and outcome, only the ones which match your proposed activity. Annex A provides guidance on relevant outputs and outcomes, including definitions and evidencing requirements. It is possible to identify other relevant outputs and outcomes, especially if they will demonstrate significant value, but these should be discussed with the Agile Cymru Team first. 

We will record outputs and any immediate outcomes as part of the claim process; the evidence required will be proportional to the grant awarded, for example self-declaration may be appropriate in many cases. We will also contact you six months after the activity is complete or at an interval relevant to the project to identify any further outcomes. The data collected will also inform any evaluations of the impact of Agile Cymru and you may be contacted for the purposes of evaluation, for example to explore more qualitative views of impact.

Potential outputs

• Number of engagements with international and UK partners.
• Number of formal networks joined.
• Number of applications for competitive funding.
• Number of participants trained.
• Number of new collaborations.
• Number of feasibility studies completed.
• Number of pilot projects completed.

Potential outcomes

• Number of existing international or UK partnerships maintained.
• Number of new international or UK partnerships established.
• Number of new formal networks created.
• Amount of competitive funding awarded (£s)
• Number of inward investment opportunities identified.
• Increased knowledge, understanding and expertise of Welsh participants.
• Number of new products, processes or services adopted or created.

Definition

An engagement refers to an interaction with one or more international and/or UK partners to exchange information about each other's organisation, identify common aims, and explore opportunities for, and the willingness to, work together (see 'number of new collaborations' as the natural progression of such activity).

The aim is to develop new partnerships or retaining/building on existing relationships that have potential to lead to greater, more sustained economic activity of significance to Wales.

Activity could include engagement through formal meetings, participation in networks and/or contact with key partners at formal events/conferences, in person or virtual.

Unit of measurement

Engagement at organisation level.

The unit of measurement is ‘engagements’ and we recognise that some projects may involve multiple interactions between the same partners (e.g. a series of meetings or workshops); however, given the scale and duration of Agile Cymru projects the collection of interactions during the project will count towards a single engagement with each partner.

For projects involving only one Welsh organisation (the majority of Agile Cymru projects), the calculation of the number of engagements created by a project is simply a count of number of international and/or UK partners.

For projects involving more than one participating Welsh organisation, an engagement can be counted between each Welsh organisation and international/UK partner in the project. The calculation of the number of engagements created by a project is a count of number of international partners multiplied by the number of Welsh organisations participating in the project.

Examples:

1. Welsh organisation A engages with International Partner 1, International Partner 2 and UK Partner 3. 
   Number of engagements = 3 (Only one Welsh organisation, so count of International/UK partners is 3)
2. Welsh organisation A and Welsh organisation B engages with International Partner 1, International Partner 2 and UK Partner 3.
   Number of engagements = 6 (Two Welsh organisations and three International/UK partners, so 2*3 = 6)

Suggested evidence

The Agile Cymru application form details key partnership(s) facilitated through the project activity (Section 1). Confirmation that these and other engagements have taken place will be by self-declaration through the claim reporting process.

Related data

Engagements must be split between UK and International partners. 

Where appropriate target region, thematic area, sector to be captured through the application process and confirmed in narrative of claim process.

Potential outcomes

• Number of new international or UK partners
• Number of existing international or UK partnerships maintained
• Number of new formal networks created

Definition

This involves joining an existing, recognised formal network or special interest group, a structure or system that brings together relevant stakeholders across borders to share information, pool resources and build capacity, and enable cooperation to access funding and advance shared aims.

Evidence suggests that being on the right network leads to a number of opportunities, therefore there is an indirect link to funding and investment outcomes.

This is separate to informal ‘building my network’ engagement, this is to capture the joining of formal networks (e.g. Vanguard etc). The final decision of whether a network is valid for this output rests with the Agile Cymru Team.

Unit of measurement

Number of networks/special interest groups joined.

Suggested evidence

Networks joined will be detailed through the Agile Cymru application form and claim reporting process.

Related data

Where appropriate target region, sector, thematic area to be captured through application process and confirmed in claim reporting process.

Potential outcomes

• Amount of competitive funding awarded (£s)
• Number of inward investment opportunities identified 

Definition

Applications to all UK and international funding sources are eligible (excluding Welsh Government). Can also include resubmissions for funding. 

This could be linked to Agile Cymru directly funding the development of bids, or could stem from other activity such as engagements and networks.

Whilst funding bids do not necessarily have to be collaborative there must be a demonstrable logic flow between the Agile Cymru activity/co-operation and the funding bid.

Unit of measurement

Number of applications submitted.

Suggested evidence

Evidence of submission of bid will be required, e.g. confirmation of receipt of the application(s) from the funding body.

Related data

Where appropriate target region, sector, thematic area to be captured through application process and confirmed in claim reporting process.

Potential outcomes

• Amount of competitive funding awarded (£s)

Definition

This relates to training directly funded through Agile Cymru. 

This could include for example specialist training around particular geographical requirements, thematic areas or funding routes. It would be expected to contribute to Agile Cymru’s objectives, increasing the chance of positive outcomes in relation to cross border and international economic co-operation.

Unit of measurement

Participants trained.

Suggested evidence

Self-declaration through claim reporting process confirming the number of participants trained. If significant funding is involved, further evidence may be required.

Related data

Where appropriate target region, sector, thematic area to be captured through application process and confirmed in claim reporting process.

Potential outcomes

• Increased knowledge, understanding and expertise of Welsh participants.
• Number of new products, processes or services created

Definition

Working with new/existing UK and International partners (excluding Welsh partners) to combine resources/expertise/data etc in order to collaborate on a new project.

Unit of measurement

New collaborations.

Suggested evidence

There will be varying levels of evidence depending on the scale of collaboration. Evidence could include:

• Email exchanges setting out details of the collaboration, including each party’s role and the shared aims and objectives. This would need to include confirmation that the collaboration has started; or
• Copy of formal / informal collaboration agreement (e.g. Memorandum of Understanding or a contract) which confirms what each party’s role is in the collaboration, the shared aims and objectives, and timescales.

Related data

Where appropriate target region, sector, thematic area to be captured through application process and confirmed in narrative of claim process. 

Potential outcomes 

Potential to contribute to a wide range of outcomes depending on the nature of the collaboration.

Definition

This output is intended to capture feasibility studies supported directly through Agile Cymru funding. 

Suggested evidence

A copy of the final report.

Unit of measurement

Feasibility studies completed.

Related data

Where appropriate region, thematic area, sector to be captured through the application process and confirmed in narrative of claim process.

Potential outcomes 

• Number of inward investment opportunities identified
• Increased knowledge, understanding and expertise of Welsh participants

Definition

This output is intended to capture pilot projects supported directly through Agile Cymru, such as a project planned as a trial, designed to test a given approach over a limited period of time, intended to lead to future activity in the field/discipline or sector.

Unit of measurement

Pilot projects completed.

Suggested evidence

End of pilot project report detailing findings such as baseline evidence and proof of concept.

Related data

Where appropriate region, thematic area, sector to be captured through the application process and confirmed in narrative of claim process.

Potential outcomes

• Number of new products, processes or services adopted or created
• Number of inward investment opportunities identified

Definition

A continued/renewed commitment to work on shared areas of interest in relation to economic co-operation.

Direct outcome from engagement activity with ‘number of engagements with international or UK partners’ but could also flow from other project activity.

Unit of measurement

Number of Partnerships maintained.

Suggested evidence

Email exchanges setting out details of plans/intentions to work together, including shared aims and objectives. 

Related data

Partnerships must be split between UK and International. Where appropriate, region, thematic area, sector to be captured through the application process and confirmed in narrative of claim process.

Definition

A commitment to work with new International and/or UK partners on shared areas of interest in relation to economic co-operation.

Direct outcome from ‘number of engagements with international or UK partners’ but could also flow from other project activity.

Unit of measurement

Number of Partnerships established.

Suggested evidence

Email exchanges setting out details of plans/intentions to work together, including shared aims and objectives. 

Evidence that it is a new partnership through the Agile Cymru application form and claim reporting process or self-declaration.

Related data

Partnerships must be split between UK and International.

Where appropriate, region, thematic area, sector to be captured through the application process and confirmed in narrative of claim process.

Definition

Linked to ‘Number of engagements with international and UK partners’ but where a sustained period of activity results in a new structured network being formalised through its membership and shared aims and objectives.

It would be expected to add value to existing networks.

Unit of measurement

Number of formal networks.

Suggested evidence

Confirmation of the existence of agreed Terms of Reference for the new Network.

Related data

Where appropriate, region, thematic area, sector to be captured through the application process and confirmed in narrative of claim process.

Definition

All UK and international funding sources are relevant (excluding Welsh Government) but there would need to be a logical link to the activity funded by Agile Cymru.

Unit of measurement

Funding awarded (£s),

Suggested evidence

A copy of the award letter or evidence of formal publication of funding award, for e.g. publication on funders website.

Related data

Where appropriate, region, thematic area, sector to be captured through the application process and confirmed in narrative of claim process.

Definition

This outcome counts the number of specific inward investment opportunities identified, rather than the amount of inward investment received, i.e. the early stages of the investment process. There would need to be a logical link to the activity funded by Agile Cymru.

Unit of measurement

Number of investment opportunities.

Suggested evidence

Identification of specific inward investment opportunities through the claim process or subsequent self-declaration. 

Related data

Where appropriate, region, thematic area, sector to be captured through the application process and confirmed in narrative of claim process.

Definition

This outcome would naturally flow from outputs such as training activity (which will increase the knowledge of participants) but could also be a result of a range of other engagement activity, feasibility studies, or pilot projects (which will be relevant at an organisational level).

Unit of measurement

Number of Welsh participants.

Number of organisations.

Suggested evidence

Self-declaration through the Agile Cymru application form and claim reporting process.

Related data

Thematic area and sector to be captured through the application process and confirmed in narrative of claim process. 

Definition

New products, processes, services adopted or created as a result of Agile Cymru activity. Would naturally flow from activity such as pilot projects, or training, also collaborations and long-term engagement where a new approach/technology/technique may have evolved. ‘New’ can be both new to the relevant organisation, e.g. adopting a best practice process, or newly created by the relevant organisation.

Unit of measurement

New products, processes, or services.

Suggested evidence

Self-declaration detailing the new product/process/service adopted or created by the relevant organisation and how it emerged from the Agile Cymru funded activity. Where appropriate this should include links to public information on the new product/process/service.

Related data

Must be split between number adopted and number created.

Where appropriate, region, thematic area, sector to be captured through the application process and confirmed in narrative of claim process.