Effective regional policy evidence review

This paper provides a high-level overview of some of the key themes from literature on effective regional policy, covering the following main areas: Aims and objectives of development policy, approach to investment, extent of decentralisation, capacity, collaboration & evaluation and monitoring.

This review begins with a brief overview of some of the key debates around development policy

Spatially blind vs place-based

A central argument in the literature revolves around whether or not economic development policy should be concerned that some regions lag behind others in terms of their development and attempt to ameliorate those differences.

“Spatially blind” approaches essentially argue that economic activity will inevitably be geographically unbalanced and, as a result, it is inefficient for policymakers to attempt to go against the grain of this process (Gill 2011).

This view depends on the notion of agglomeration economies. This is the idea that firms and workers tend to be more productive in large, densely populated areas (Gill 2011, Barca 2011 OECD 2018b, Nguyen 2019). Because of this, economic activity is likely to always be somewhat geographically unbalanced in favour of these agglomerations and policy initiatives are unlikely to reverse this. Instead policy should focus on making the most of centres of economic growth wherever they exist. This typically means efforts to ensure agglomerations function effectively and investments that help firms and individuals living in less developed areas to access the opportunities found in large cities. This might mean things like investments in infrastructure to enable people to move or commute to these areas or general investments in skills to access skilled work concentrating in cities. Beyond this, public policy should focus on the provision of general, population wide services that are available to people irrespective of where they live.

This is often put in terms of helping “people” rather than “places” – the objective of policy should be to help people access economic opportunities wherever those opportunities exist, not to spread economic opportunities more equally across areas.

Broadly speaking there is good evidence for the notion that increasing urban concentration is associated with increasing productivity and economic growth although the exact size of the effects vary considerably across studies and countries (e.g. Melo et al 2009, Frick & Rodriguez-Pose 2016, OECD 2018b). In addition Gill (2011) argues that, historically, development policy efforts aimed at achieving even levels of economic growth have been much less successful than development policies oriented around agglomerations However, the literature also notes a number of reasons to be cautious about a purely spatially-blind, agglomeration driven approach. In particular:

• Weaker performing regions still contribute to national economic performance, by extension improving the performance of these regions is likely to support broader economic growth while failing to support them is likely to hold back aggregate economic growth (OECD 2012, Gal & Egeland 2018).
• Urban regions are not necessarily always the fastest growing. For example, between 1995-2007 rural regions in OECD countries grew faster than urban regions (OECD 2012). As such, an excessive focus on urban areas may miss opportunities to promote growth
• Broader based economic growth is likely to make countries less vulnerable to economic shocks and reduce the impact of geography on individual life chances (OECD 2012)
• While Gill (2011) notes the apparent success of Ireland’s largely agglomeration-driven, spatially blind approach, in the UK cities appear to have been less successful at generating growth outside of London and the South East (McCann 2016, Pike 2018).
• Agglomeration can also have negative effects. The potential costs to promoting the growth of already large urban areas include things like pollution and traffic congestion
• There are obvious moral and political implications to allowing some areas to be “left behind” while focussing development efforts on major cities (Hendrickson et al 2018, Rodriguez-Pose 2018)
• It should also be noted that, compared to other parts of the UK, Wales is starting from a fairly low base in terms of existing agglomerations (Brill et al 2015), potentially limiting the scope for agglomeration-oriented policies in Wales.

As a result, there appears to be a reasonable level of support in policy circles for a more place-sensitive approach to development policy that does, to some extent, attempt to reduce or ameliorate geographic disparities in terms of development (e.g. OECD 2018b, Hendrickson et al 2018).

It should be noted that a more place based or place sensitive approach to development policy does not preclude efforts to harness the benefits of agglomeration economies (Barca 2011). Indeed, the OECD (2018b) advocates both a place-based policy and emphasises that well-functioning cities are a key characteristic of regions of less developed regions that have caught up with more developed regions. However a more place-based approach to agglomeration may imply a broader range of approaches than helping large agglomerations grow further, for example looking at the scope for developing networks of smaller agglomerations (Barca 2011) and paying close attention to the mechanisms by which growth in agglomerations can be transmitted to other areas (OECD 2018b, Pike 2018).

Convergence vs making most of potential

One key criticism of historical regional development policy noted above is that while the overarching objective of regional has often been to reduce disparities between areas, this has frequently not happened (although as the OECD (2012, 2018b) notes, some degree of “catching-up” is possible).

However, this does not necessarily mean that regional policy has had no impact. A policy can be said to have an “impact” if a change occurs that would not have occurred if that policy had not been implemented. Regional development policies may have an impact (in the sense that they enable regions to grow faster than they would have in the absence of a regional development policy) but regional inequalities could still remain the same or even grow depending on what is happening in other regions. It should also be noted that regions may grow or “catch up” with wealthier regions for reasons that are entirely unrelated to policy.

Consequently, assessments of the value of regional policy are increasingly focussing more on whether they enable regions to develop faster and to a greater extent than they would have in the absence of these activities, whether policies enable regions to make the most of their growth potential (OECD 2012).

A final point to note is that establishing the impact of regional development policy in this sense is more complicated than comparing headline indicators of GVA between regions. As such, the assessment of regional policies requires good quality evaluation procedures, something that will is discussed in more detail in a later section.

Broad sectoral priorities

Identifying specific sectors or areas of investment is beyond the scope of this paper. However it is useful to consider some broad frameworks that outline different ways of targeting investment.

There are divergent views in the literature on what kinds of economic sector development policy should focus on. Three major approaches are worth discussing:

Firstly, a knowledge economy approach, which emphasises investments in innovative, technology-intensive, research & development led industries, for example high value manufacturing, life sciences, information and communication technology.

Secondly, the OECD (2018b) argues for a focus on tradable sectors to improve regional productivity on the basis that:

• Regions that have been “catching up” with leading regions typically have large and growing tradable sectors
• Regions which relied on non-tradable sectors prior to the financial crisis were hit hardest by the crisis and have found it difficult to recover
• Because firms in tradable sectors can potentially sell their goods to a market beyond their immediate area they have greater potential for growth
• Firms in tradable sectors are more exposed to international competition so face greater incentives to innovate and improve their productivity
• Empirical evidence suggests that productivity improvements in tradable sectors are more likely to come from within-firm improvements in productivity. In non-tradable sectors productivity improvements are more likely to come from shifts in employment from less to more productive firms
• It is highly likely that there will be some overlap between tradable and knowledge economy sectors, most knowledge economy sectors will also be tradable sectors however, tradable sectors are likely to be significantly broader

The OECD does not suggest that investments in non-tradable sectors should be entirely ruled out:

• Non-tradable sectors such as health services and construction provide necessary and useful services and products. The outputs from these sectors may help underpin the growth of tradable sectors
• However, decisions about investments involving non-tradable sectors (and assessments of their effectiveness and impact) should not be based on their direct economic impact (for example, in terms of jobs created) but on the value of the services they provide as well as their contributions to growth in other areas

Finally, a foundational economy approach, which emphasises support to industries that “support everyday life” by creating and distributing goods and services that are consumed by all (Bentham et al 2013, Brill et al 2015, Froud et al 2019). Key examples include food retail and production, health and social care, utilities and transport and distribution. The broad argument for a foundational economy approach is as follows:

• Because foundational economy sectors provide good and services that support everyday life, strengthening and improving these sectors will have a direct impact on people’s well-being
• Foundational economy sectors contribute significantly to employment. As a result, improving job quality and wages in these sectors will have a wider direct impact than the development of “knowledge economy” sectors, which are only likely to employ relatively small numbers of people.
• Foundational economy sectors are potentially more “resilient” than other economic sectors. Populations always need the goods and services produced by the foundational economy and so demand is unlikely to fluctuate to the same extent as other sectors.
• Government has more “levers” to influence the foundational economy. Much of the foundational economy is directly controlled by the public sector (for example health services). In other areas, the public sector is a major purchaser of foundational economy services (for example in social care) meaning there is scope to shape firm behaviour via procurement policies.
• Foundational economy advocates are arguably more sceptical that the interests of government, employers, employees and consumers will necessarily and inevitably align.

As far as we are aware, there is no clear evidence base that would enable us to say which of these approaches is “better”. In particular, given the novelty of the foundational economy approach, there is limited evidence on whether its policy prescriptions yield the kind of benefits its advocates expect. However, it is not necessarily the case that the different approaches are incompatible. As already noted, knowledge economy-type sectors are likely to be tradable sectors. Foundational economy advocates do acknowledge the value of investments in more knowledge intensive industries (Bentham et al 2013).

To some extent the choices between these approaches depends on the broad objectives of regional policy. Both the knowledge economy and tradeable sectors approaches are oriented towards stimulating aggregate economic growth and, particularly in the OECD’s tradable sectors approach, productivity growth. The assumption here is that targeting these objectives will create conditions for broader increases in well-being and prosperity. The foundational economy approach, however, implies a focus on the more direct impacts policy can make to broader well-being.

Setting objectives

Whatever the objectives of regional investment policy are, there is a clear recommendation from the literature that these are made clear at the outset (OECD 2018a). It is beneficial for policy goals to be defined collectively and to ensure there is consensus on those outcomes across the broad range of actors involved in regional development policy and that when designing policy, the relationship between inputs, outputs and outcomes is clearly articulated (OECD 2018a).

Tailoring to regional needs and characteristics

There is broad support in the literature for the idea that there is not a simple recipe for success in regional development policies that can be applied to all regions and that what works in one area, might not work in another. Instead, policies need to be tailored to meet the particular needs and complement the particular strengths of individual regions (Barca et al 2012, Crescenzi et al 2017, OECD 2018a, OECD 2018b, Di Cataldo & Monastiriotis 2018, Rodriguez-Pose & Wilkie 2018):

• Focusing on the UK, Di Cataldo & Monastiriotis (2018) find that EU funds are most conducive to growth when they have been used to address areas that regions “lack” – for example infrastructure spending appears to be more effective in regions where infrastructure is relatively underdeveloped
• Fratesi & Perucca (2014) find that the impact of EU funds is mediated by the existing characteristics and assets of a region (which they describe as territorial capital) of a particular region, so for example, labour market policies are more effective where regions have higher levels of human capital
• Looking across countries Crescenzi et al (2017) find that EU funds have performed better in regions where spending is aligned with perceived needs
• Using comparative case studies across a range of countries Rodriguez-Pose and Wilkie (2018) identify a range of examples where non-spatially targeted interventions have been “wasteful” (for example blanket investments in infrastructure) and other where more careful tailoring has been more successful (for example, infrastructure investments targeted at alleviating specific bottlenecks)
• Drawing on case study evidence from across the OECD area, the OECD (2012) suggests regions should focus on building on their existing strengths and potential

To give some examples of what this might mean in practice:

• Ensuring that investments in infrastructure are clearly targeted at alleviating specific regional bottlenecks, avoiding blanket or generic road building schemes. Attention should also be paid to the overall level of infrastructure in a region - there is a possibility that general infrastructure improvement investments have diminishing returns for regions that already have a reasonable level of connectivity (Rodriguez-Pose & Wilkie 2018)
• Basing decisions about which sectors to support on current regional strengths and assets. For example, identifying sectors that are related to industries that already have a strong presence in the region or identifying location-specific features (for example geographic features) that are useful for particular industries (OECD 2018b)
• Ensuring that decisions about skills development take account of local demand for skills, failure to do so means training participants may not be able to find work that makes best use of their skills or could leave the region for better opportunities elsewhere (Rodgriguez-Pose & Wilkie 2018)
• In relation to policies to promote research and development, ensuring that regional economies are able to translate and make use of the outputs of R&D activity in practice. Where they do not, R&D policies may be successful in their own terms (e.g. more R&D activity might take place) but will not contribute to wider economic growth. This might mean ensuring that R&D activity is related to the activities of firms in the region and also considering the capacity

Multiple, complementary policy areas

The literature also suggests that regional development policy needs to cover multiple policy areas rather than concentrating efforts on individual areas (e.g. infrastructure, innovation, skills development) (Fratesi & Rodriguez Pose 2004, OECD 2012, Crescenzi & Giua 2016, Rodriguez-Pose & Wilkie 2018, Di Cataldo & Monastriotis 2018). In addition, activity across different policy areas needs, as far as possible, to be complementary and co-ordinated (Fratesi & Rodriguez-Pose 2004, OECD 2012, OECD 2018b, Rodriguez-Pose & Wilkie 2018). A historical criticism of regional investment policy is that it had a tendency to rely on isolated policy areas to deal with challenges that are often multi-dimensional, for example assuming that investments in areas such as human capital or infrastructure will contribute directly to growth in and of themselves (Rodriguez-Pose & Wilkie 2018). A key risk is that activity in one area will be undercut by a lack of activity in another. As an example, the OECD (2012) notes that in the absence of policies to stimulate business growth, human capital investment policies may lead to “brain drain” as skilled individuals move out of a region to find work elsewhere. At the same time, businesses may not be able to grow without access to skilled workers.

One caveat to this is that, while action in multiple areas is beneficial, the OECD (2012) notes that there is a risk of a proliferation of small projects that are too thinly spread to have meaningful impact. As such, there is a need to balance the requirements of a wide-ranging package of development measures with projects of sufficient size.

Experimentation

Given that it is not straightforward to know exactly what will work in particular areas, there is also general support for an “experimental” approach to development policy where different ideas are tried and tested (OECD 2018a, Hendrickson et al 2018). However, for this to be effective there is a need for a well-developed monitoring and evaluation system to assess how well new and innovative ideas are working (OECD 2018a).

A further question is the extent to which control over regional development policy should be decentralised. Decentralisation is broadly taken to mean transfer of decision making power away from the centre. This is in contrast to “deconcentration” where responsibility for administering and implementing policies is transferred from the centre but decision making power remains centralised (Martinez-Vasquez et al 2015).

A place-based approach to regional development leans towards the idea of decentralisation, with regions making decisions about policies to be pursued in their area. The literature offers a number of arguments favouring the devolving of responsibilities to the subnational level.

As noted above, a key element of a place based approach is that effective development policy needs to be tailored to local needs. To tailor policy to local needs effectively, policymakers need to draw on the knowledge of a wide range of actors. Central government alone cannot always determine the best course of action in all instances and local actors might have a better understanding of the needs of particular areas (Rodriguez-Pose & McCann 2011, Barca et al 2012, Di Cataldo & Monastiriotis 2018).

This chimes with arguments from other bodies of literature. For example, the economic theory of fiscal decentralisation suggests decentralisation will enable decisions to be better aligned with the preferences and needs of the local population (Oates 1972 cited in Martinez Vasquez et al 2015). Advocates of experimental governance generalise this further and argue that due to increasingly complex socio-economic challenges, central governments need to involve a wider range of actors in problem solving processes across all policy areas (Sabel and Zeitlin 2012 cited in Morgan 2018).

Linked to the above, decentralisation may allow for a more experimental approach to policy making in regional development by bringing new voices and knowledge into the policy process and providing space to try innovative schemes on a smaller scale (Morgan 2018). Other benefits identified include the idea that decentralisation may help foster competition between areas (for example for investments by businesses and to attract skilled workers), which will encourage better policy making (Martinez-Vasquez & McNab 2003).

The OECD (2018a) also argues that it is important that local areas feel they have “ownership” over development strategies in their area, something that may be more likely where those areas have meaningful control over those strategies. Similarly Rodrigez-Pose and McCann (2011, also Barca et al 2012) emphasise the importance of a sense of “community” to regional development policy as a way of ensuring broad support for efforts to promote economic development, something that they argue is more likely to be the case at local levels.

However, there may also be reasons to be cautious about devolving responsibilities:

• There are concerns that decentralisation may exacerbate regional inequalities. In particular, given the association between administrative capacity (see subsequent sections) and effective regional policy there are concerns that decentralisation might favour more developed regions with higher levels of capacity.
• There is a risk of fragmentation, complexity and duplication of activities across areas (OECD 2018b)
• While competition between geographic areas can be productive, there is also a risk of unhealthy competition between devolved areas that prevents different regions from pursuing policies that could be mutually beneficial or even that one region may undertake interventions that actively harm another (OECD 2018b).
• Smaller government units may not be large enough to affect change of sufficient size and scale (Morgan 2018, Allain-Dupre 2018)

It should also be noted that, for many of the arguments presented above, maximising the purported benefits of decentralisation implies a kind of "double decentralisation" with more decisions being made at the local level but also efforts to involve a wider range of stakeholders and ordinary citizens (OECD 2018a, Morgan 2018, Allain-Dupre 2018). If the place based argument that development policy requires a broad range of knowledge from different actors to be effective is taken seriously, this implies a need to ensure many different voices are heard in the policy making process (Rodriguez-Pose & McCann 2011). If decision making at the local level is dominated by local "elites" (for example, political and business leaders) there is a risk that strategies may be too narrow and perhaps economically focussed rather than considering broader social goals (Morgan 2018). In addition, just because political leaders feel ownership over development activity in their area, it does not follow that this will also be shared by the broader population (OECD 2018a). This risks undermining the legitimacy of development strategies.

Empirical evidence on the links between decentralisation and growth

In terms of empirical evidence, a voluminous literature on the potential impacts of fiscal decentralisation (e.g. transferring decisions about how government money is spent and/or raised away from the centre) provides virtually no clear answers as to whether decentralisation is likely to enhance economic growth or not. Reviews of this literature typically report that individual studies can vary between finding a clear positive association between decentralisation and growth, a clear negative association and no evidence of a link at all (Blochliger & Egert 2013, Martinez-Vazquez et al 2015, Baskaran et al 2016).

Having said this, Baskaran el al’s (2015) review does find stronger evidence that decentralisation may contribute to positive outcomes in areas other than overall economic growth, in particular:

• Reducing inequality between geographic areas
• Reducing individual/household inequality
• Improving some forms of public service

The first of these is particularly worth noting given concerns that decentralisation may exacerbate inequalities. Blochliger & Egert (2013) also draw attention to some important regularities in the literature, for example:

• There is stronger evidence of an association between decentralisation and growth in high income countries than low income countries
• How decentralisation is measured/defined is important, in particular studies measuring decentralisation in terms of control over revenue tend to find a more robust link than studies which define it in terms of control over spending
• Single country studies tend to find stronger relationships than cross country studies

The two latter points suggest that the success of decentralisation may depend to a large extent on the context and specific arrangements that are adopted in a particular territory, in other words decentralisation in itself may not be a good or bad thing, rather it can have advantages and disadvantages depending on the how and where it is implemented.

In terms of policy effectiveness, Morgan's (2018) review of evidence on experimental democracy identifies cases where sub-national governments have been successful in developing innovative approaches. However, Morgan also notes evidence that these small-scale programmes can have limited impact and that in some areas, central government action can be more important. For example, in the US meaningful progress on anti-poverty programmes has typically come from central rather than local government action (Super 2008 cited in Morgan 2018).

Similarly, Crescenzi & Giua (2016), while finding evidence broadly in favour of a "bottom-up" approach to regional development policy also suggest there may still be a case for more "top-down" and spatially blind funds, in particular to support regions that may lack capacity to make the most of decentralised funds.

Managing decentralisation

Building on the idea that the key question is how decentralisation is implemented Allain-Dupre (2018) identifies a number of key challenges that need to be overcome in allocating responsibilities to sub-national levels of government. In particular, Allain-Dupre notes issues around:

• Unclear assignment of responsibilities between different levels of government
• Resource and capacity bottlenecks – points in the system where resources or capacity are insufficient for that part of the system to fulfil its responsibilities effectively. A particular issue is the devolution of responsibility without adequate resource
• Co-ordination challenges – both vertically (e.g. between local and national government) and horizontal (e.g. between local governments in different areas)
• Identifying the correct scale for public investment and service delivery (for example balancing the potential benefits of having decisions made “closer” to the people versus the benefits of action on a large, national scale)

There are not clear answers to how to tackle these challenges, however Allain-Dupre identifies a number of guidelines to help shape decisions including:

• Ensuring that the responsibilities of different levels of government are clearly defined and that powers are decentralised in a coherent way
• Ensuring there are sufficient resources and capacity at the sub-national level (in particular undertaking capacity building – dealt with in subsequent sections and ensuring resources align with responsibilities)
• Ensure there are mechanisms in place for co-operation and collaboration between levels of government (e.g. national and regional) and between different geographic areas (e.g. different regions)
• Undertake complementary reforms (e.g. over the governance of land-use, citizen participation and innovative public-service delivery)
• Enhance data collection and strengthen performance monitoring (evaluation and monitoring are dealt with in more detail in subsequent sections)

Insufficient capacity at local levels to manage the demands of decentralised regional policy is considered to be a particular risk (capacity issues are discussed in more detail in the next section). To manage this, a number of authors (e.g. Allain-Dupre 2018, OECD 2018a), discuss the scope for “asymmetrical decentralisation” – where different regions are offered different levels of responsibility depending on assessments of their ability to manage additional powers. This is seen as a method for dealing with uneven levels of capacity across regions. However, asymmetrical decentralisation comes with its own risks, in particular there are concerns that allowing “strong” regions more powers will exacerbate regional inequalities (although this theory does not appear to have been tested in the empirical literature). Other risks include increasing complexity and a detrimental impact on relationships between national and regional governments (for example if regions awarded less autonomy feel dissatisfied with this state of affairs).

As such authors emphasise the importance of ensuring decisions about asymmetrical decentralisation are as transparent as possible, that the process is subject to constant monitoring and evaluation and that the level of decentralisation is Regional Investment Steering Group for Wales treated flexibly and experimentally – allowing for adjustments to be made depending on the findings from monitoring and evaluation (Morgan 2018).

Conditions and incentives

In an effort to mitigate the potential downsides of devolution/decentralisation, it is relatively common for national (or supra-national) governments to put in place systems of incentives and conditions attached to decentralised funds. The OECD (2018a) identifies several things to consider when developing these systems:

• Incentives associated with a regional policy framework should be considered systematically to avoid perverse incentives/encouraging one behaviour while expecting another
• Incentives should be attached to goals that those responsible for implementing policies are in control of. However, it is vital that those goals are clearly linked to the broader outcomes the policy seeks to achieve. There are 2 risks here:
  • Focussing on policy outputs (e.g. number of participants completing an employability scheme) without considering whether those outputs are linked to the desired outcomes (e.g. whether the employability scheme leads to participants finding work)
  • Attaching incentives to broader outcomes that are affected by factors other than policy and are, therefore, not in the control of those implementing the policy
• There is a need to be wary of the risks of focussing excessively on extrinsic motivation and designing a system of incentives that crowds out intrinsic motivation
• Those responsible for achieving goals should be involved in setting those goals
• Effective feedback mechanisms are important to the functioning of incentive systems
• Systems of conditions and incentives work best when they are underpinned by strong, trusting relationships

The “capacity” of different levels of government has been identified as a key factor in the successful implementation of regional policy In the context of regional policy, capacity broadly refers to the ability of a national or sub-national government to develop, plan, implement and evaluate effective development policies Different dimensions of capacity can include (Smeriglio et al 2016, OECD 2018a):

• The skills and abilities of staff (including the leadership abilities of those at the top of the organisation)
• Financial factors (e.g. access to stable and appropriately sized budgets and other sources of finance)
• Institutional arrangements (e.g. arrangements for co-ordination across levels of government, stakeholders, geographical and policy areas)
• Policy processes (e.g. procurement and project selection processes)

In terms of empirical evidence, a number of studies have found evidence of a link between quality of governance and the effectiveness of regional policy (Rodriguez-Pose and Garcilazo 2015, Smeriglio et al 2015, Rodriguez-Pose & Di Cataldo 2015, Crescenzi et al 2016). Given this, there is a general agreement in the literature that some form of capacity building needs to form a part of any regional development strategy. Issues of capacity overlap considerably with questions of decentralisation discussed elsewhere in this paper. If a decentralised regional investment policy is pursued, it is widely argued that capacity at sub-national level is crucial (Allain-Dupre 2018). This is important to note because capacity building is not always a high priority in regional development (OECD 2018a).

While much of the focus here is on the local level, it should also be noted that capacity constraints can also occur at the national and supra-national level. For example, Morgan (2018) notes that a potential criticism of recent round European Structural funds is that staffing at the EU level has been too oriented towards compliance, with too little capacity in areas of policy advice and expertise.

Given the breadth of the concept of capacity, capacity building interventions are likely to involve multiple strands, the literature notes a number of approaches and factors to consider in designing these interventions.

Formal learning and the dissemination of “best practice” may be useful to an extent but are unlikely to be sufficient on their own. Smeriglio et al (2016) note the importance of human resource management beyond training in developing capacity, in particular:

• Recruiting sufficient numbers of staff who come to the organisation with an existing level of skill
• The adoption of HR policies that encourage the retention and promotion of skilled staff

In addition to formal learning, allowance should also be made for “learning by doing” in which national and subnational governments learn what works from interacting with each other over time (OECD 2018a).

Finally, it should be noted that the capacity of a subnational government might be shaped by factors outside of its direct control (Smeriglio et al 2016) for example:

• The financial resources made available to them by central government
• The local availability of organisations providing specialist services (for example training providers and external evaluators)
• The capacity of beneficiaries of funding (e.g. private firms, other public bodies) to plan, implement and co-fund projects (Smeriglio et al 2016)
• National level rules and administrative culture

Simplification

The impact that national level rules and administration have on subnational capacity, highlight the need for simplicity in regional policy frameworks (OECD 2018a). Excessively complex or opaque frameworks can generate unnecessary burdens on, and stretch the capacity of, governments at lower levels. The OECD (2018a) identifies a number of ways in which regional development frameworks can be kept simple:

• Ensuring there are not regular changes to the rules (OECD 2018a). Aside from anything else such changes undermine learning activities designed to build capacity
• Refraining from using excessively detailed conditions to address performance issues – one temptation can be to get around capacity issues by setting very detailed rules for sub-national governments to follow. However, this risks exacerbating capacity issues
• Ensuring all information and guidance is understandable and clearly written

A cross-cutting issue that follows from a number of points above is the importance of collaboration. Regional policy requires significant levels of collaboration between multiple groups (OECD 2018a), in particular:

• Between different levels of government (e.g. national and subnational) is important in regional policy in general to ensure the activities of different tiers of government are co-ordinated and the expectations placed on different groups are clear. This is likely to become increasingly important in more decentralised systems
• Between different policy areas within particular levels of government (e.g. skills policy and infrastructure policy) – this is particularly relevant given the importance of policy synergies discussed above
• Between different regions/geographical areas, for example to facilitate larger multi-area projects that could benefit more than one region, to overcome problems of insufficient scale in particular regions and to avoid problems of disadvantageous competition and duplication of activities between regions
• Between government and external stakeholders at both the national and sub-national level, for example to draw on a range of knowledge to inform policy decisions and ensure policies align with regional needs

In terms of facilitating collaboration, a number of key messages stand out (OECD 2018a, OECD 2016):

• Quality of relationships are vital, there is a risk to focussing on procedures and formal collaboration mechanisms at the expense meaningful, trusting and co-operative relationships
• Relationships need to be built through regular and transparent dialogue
• The development of strong relationships is likely to require up-front costs but is necessary for longer term success
• Formal collaboration mechanisms need to be effective and provide participants with a motivation to collaborate
• Informal dialogues and social networks are important in underpinning formal collaboration
• Finding a balance between top down and bottom up approaches is important
• Unilateral decisions undermine trust and hamper collaboration
• At a national level, interministerial committees can be effective at facilitating working across policy areas and ensuring regional policy is streamlined and consistent. However, these arrangements only appear to have real influence where the committee meets regularly and is a permanent structure

The evaluation of regional policies is identified as a vital across the literature to understand whether policies are having their intended effect and to generate knowledge about what kinds of interventions work (e.g. OECD 2018a, OECD 2016, McCann 2016). There are a number of factors to consider when deciding on approaches to evaluation.

The purpose of evaluation and monitoring

Evaluation & monitoring can be both a tool for accountability and a tool for learning (McCann 2016). From an accountability perspective, evaluation is about assessing whether an intervention worked and, more broadly, whether those designing and implementing policy are doing a “good job”. From a learning perspective, evaluation is about generating knowledge that can be used to develop and improve policy in the future.

These approaches are not inevitably opposed but there can be tensions, for example an excessive focus on accountability and whether goals are being achieved can create a “league table culture” McCann (2016). This potentially inhibits engagement with evaluation and discourages innovation and risk-taking (McCann 2016).

It is also worth noting that different kinds of evaluations can serve different purposes. For example, process evaluations focus on how an intervention is working in practice and how well it is being implemented. Impact evaluations, on the other hand, are focussed on establishing whether an intervention brought about the changes that Regional Investment Steering Group for Wales were expected. Both generate potentially useful information from either an accountability or learning point of view but it is important to be clear about the aims of an evaluation at the outset.

The timing of evaluation and monitoring

Evaluations are often thought of activities that should take place either during or in the immediate aftermath of a particular project or investment. However different kinds of evaluation activity can occur at a range of different times in the policy cycle.

In particular "ex-ante" evaluations can take place before a project has begun, during the project development or selection phase (OECD 2018a). This activity is particularly important in relation to discussions above, about tailoring interventions to local needs and "experimental" approaches. This implies a need for proper analysis of whether policies do in fact meet the needs of a particular region and to ensure that novel approaches have clear and plausible theories of change based on available evidence and theory (Hendrickson et al 2018)

Both process and impact evaluations can potentially take place either during or after the lifetime of a project. However, the timings of impact evaluations, need to take into account the likely timeframe for the emergence of impacts (OECD 2016). In particular, the benefits of some projects may only be realised over an extended period, while others may have beneficial impacts in the short term that recede over time. This is likely to vary considerably across different projects, policy areas and impact types. In addition, process evaluations are likely to be considerably more effective if they take place while a project is operational so that data collection is not reliant on the recall of interview respondents.

Monitoring activity should typically be undertaken throughout the lifetime of a project from design through to execution (OECD 2018a). However, linking back to discussions of accountability and learning above, there is a recommendation from some authors (e.g. Sabel 2016 cited in Morgan 2018) that monitoring activity needs to be more "diagnostic". This means monitoring data is used as a tool for problem solving, identifying potential problems and developing solutions, rather than just as a way to see whether projects are meeting targets. This suggest monitoring requires not simply ongoing data collection but also ongoing analysis of the data collected.

The level and breadth of evaluation

Evaluation can take place at a number of "levels, for example individual projects or investments can be evaluated, the strategies of particular regions can be evaluated or the regional development as a whole cane be evaluated. There are advantages and disadvantages to these different approaches, for example:

• It is easier to attribute outcomes at the project level (e.g. to be more confident that particular outcomes were “caused” by a particular policy). However, these evaluations struggle to provide information on certain key areas such as how different projects cohere and the overall contribution of regional policy to regional development (OECD 2016)
• Macro, programme level evaluations can provide a better assessment of the overall impact and coherence of regional policy but may generate less useful information about which specific interventions work most effectively (WWCfLED 2016)

Consideration should be given to how evaluation can capture the extent to which policies and projects implemented by one region affect other regions, either positively or negatively (OECD 2018b).

Evaluation and capacity

A final point to note is that considerations of capacity also have a role to play in decisions about evaluation. In particular, it will be important to consider the level of resource available to pay for evaluations themselves - good evaluations require appropriate funding (OECD 2018a) and the number and skills of staff within the bodies responsible for monitoring and evaluation.

As Smeriglio et al (2016) note, capacity goes beyond the public sector bodies responsible for planning and implementing regional development bodies. In particular issues like the availability and quality of external evaluation contractors needs to be considered, as does the complexity of the overarching evaluation framework and the guidance available to help people understand there roles in that framework.

Finally, effective evaluation requires clarity about the intended goals and outcomes of policy (OECD 2018a). If the goals of a policy are unclear, it is exceptionally difficult to evaluate whether those goals have been achieved.

Allain-Dupre (2018) "Assigning responsibilities across levels of government: Trends, challenges and guidelines for policy-makers", OECD Working Papers on Fiscal Federalism, No. 24, OECD Publishing, Paris.

Barca (2011) “Alternative approaches to development policy: Intersections and divergences” in OECD Regional outlook - building resilient regions for stronger economies

Barca, et al (2012) “The case for regional development intervention: place‐based versus place‐neutral approaches” Journal of regional science, 52(1), pp.134-152.

Baskaran et al (2016) “Fiscal Federalism, Decentralization and Economic Growth: A Meta-Analysis” Freiburg Discussion papers on Constitutional Economics

Bentham et al (2013) Manifesto for the Foundational Economy

Blochliger & Egert "Decentralisation and Economic Growth - Part 2: The Impact on Economic Activity, Productivity and Investment", OECD Working Papers on Fiscal Federalism, No. 15, OECD Publishing, Paris.

Brill et al (2015) What Wales could be

Crescenzi & Giua (2016) “The EU cohesion policy in context: does a bottom-up approach work in all regions?” Environment and Planning A, 48 (11). pp. 2340-2357

Crescenzi, et al (2016) Government quality and the economic returns of transport infrastructure investment in European regions. Journal of Regional Science, 56(4), pp.555-582.

Crescenzi, et al (2017) “The EU cohesion policy and the factors conditioning success and failure: evidence from 15 regions” Regions Magazine, 305 (1). pp. 4-7

Di Cataldo & Monastiriotis (2018) “An assessment of EU Cohesion Policy in the UK regions: direct effects and the dividends of targeting” LEQs paper no 135/2018

Fratesi & Perucca (2014) “Territorial capital and the effectiveness of cohesion policies: an assessment for CEE regions” Investigaciones Regionales 29, pp 165-191

Frick & Rodríguez-Pose 2016. “Average city size and economic growth.” Cambridge Journal of Regions, Economy and Society, 9(2), pp.301-318.

Fround et al (2019) Introducing the foundational economy Presentation to the Welsh Government

Gal & Egeland (2018) “Reducing regional disparities in productivity in the United Kingdom” OECD Economics Department Working Papers, No. 1456, OECD Publishing, Paris

Gill (2011) “Improving regional development policies” in OECD Regional outlook - building resilient regions for stronger economies

Hendrickson et al (2018) - Countering the geography of discontent: Strategies for places left-behind, Brookings Institute

Keep (2016) “Improving Skills Utilisation in the UK – Some Reflections on What, Who and How?” SKOPE Research Paper No. 123

Kierzenkowski et al (2017) “Where to get the best bang for the buck in the United Kingdom? Industrial strategy, investment and lagging regions”, OECD Economics Department Working Papers No. 1426

Martinez-Vazquez & McNab, (2003) Fiscal decentralization and economic growth. World development, 31(9), pp.1597-1616.

Martinez-Vasquez et al (2015) “The impact of fiscal decentralization: A survey” GEN Working Paper A 2015 – 5

McCann (2016) The UK Regional-National Economic Problem: Geography, globalisation and governance

McCann & Rodríguez-Pose (2011). “Why and when development policy should be place-based” OECD Observer, 14(2011), pp.203-213.

Melo et al (2009) “A meta-analysis of estimates of urban agglomeration economies” Regional Science and Urban Economics, Volume 39, Issue 3, Pages 332-342

Morgan (2018) Experimental governance and territorial development OECD Seminar series: Broadening innovation policy – New insights for regions and cities

Nguyen (2019) “Regional Economic Disparities and Development in the UK” NIESR Policy Paper 010

Oates (1972) Fiscal Federalism

OECD (2012) Promoting growth in all regions

OECD (2016) Regional Outlook: Productive regions for inclusive societies

OECD (2018a) Rethinking regional development policy-making

OECD (2018b) Productivity & jobs in a globalised world: How can all regions benefit?

Pike (2018) The limits of city centrism, Centre for Urban and Regional Development Studies (CURDS), Newcastle University 

Rodríguez-Pose (2018)  “The revenge of the places that don’t matter (and what to do about it)”. Cambridge Journal of Regions, Economy and Society, 11(1), pp.189-209.

Rodriguez-Pose et al (2007) “Fiscal decentralisation,efficiency, and growth” IMDEA working papers series 2007/11

Rodriguez-Pose & Fratesi (2004) “Between development and social policies: the impact of European Structural Funds in Objective 1 regions” Regional Studies 38, 97–113

Rodríguez-Pose & Di Cataldo, “Quality of government and innovative performance in the regions of Europe”, Journal of Economic Geography, Volume 15, Issue 4, Pages 673–706,

Rodríguez-Pose & Garcilazo, (2015) Quality of government and the returns of investment: Examining the impact of cohesion expenditure in European regions. Regional Studies, 49(8), pp.1274-1290.

Rodríguez-Pose & Wilkie (2018) “Strategies of gain and strategies of waste: What determines the success of development intervention?”. Progress in Planning.

Sabel (2016) How to make smart specialisation work Keynote speech, Smart Regions conference Brussels

Sabel & Zeitlin (2012) “Experimentalist governance” in Levi-Faur (ed) The Oxford Handbook of Governance

Smeriglio et al (2015) Administrative capacity building and EU cohesion policy EIBURS project Paper 1: Literature review

Smeriglio et al (2016) - Administrative capacity and cohesion policy: New methodological insights from Italy and Poland in Dotti, N.F., (ed.), Learning from implementation and evaluation of the EU Cohesion Policy: Lessons from a research-policy dialogue. RSA Research Network on Cohesion Policy, Brussels.

Super (2008) “Laboratories of destitution: Democratic experimentalism and the failure of antipoverty law” University of Pennsylvania Law Review 157 p541-616

What Works Centre for Local Economic Growth (2015) Transport

What Works Centre for Local Economic Growth (2016) Area-based initiatives: EU Programmes