Measuring and monitoring democratic health: a Delphi study (summary)

Background and research aims

Many modern democracies are experiencing a widening gap between citizens and the institutions designed to represent them ('A Crisis of Political Trust? Global Trends in Institutional Trust from 1958 to 2019' Valgarðsson et al, 2025). This has resulted in widespread dissatisfaction with political systems, declining trust in institutions, and escalating social and political divisions. Factors such as economic hardship, perceptions of corruption, inequality, and crises like the COVID-19 pandemic have all deepened these divides ('Global Public Opinion in an Era of Democratic Anxiety' Wike and Fetterolf, 2021). Recent research has shown that although representative democracy continues to enjoy broad support, dissatisfaction with how it functions has grown significantly in recent years. A median of 59% of people across 24 countries expressed dissatisfaction with democracy, citing that elected officials are increasingly out of touch and political parties are failing to represent their views effectively ('Representative Democracy Remains a Popular Ideal, but People Around the World Are Critical of How It’s Working' Pew Research Center, 2024).

A healthy democracy is fundamental to a functioning and representative society, ensuring accountability, checks on power, the protection of rights, and a generally more stable society ('The importance of democracy' Chatham House, 2024). However, to determine whether a country has a healthy democracy and to identify where to focus efforts to strengthen it, there must be a mechanism for measuring and monitoring democratic health. This mechanism should capture the full complexity of democracy and civic engagement, going beyond simplistic metrics like voter turnout.

Currently, there are international measures for democratic health related to the UK; however, these are not disaggregated to a national level, so they cannot reflect the specific context within Wales. As a result of this, the Welsh Government commissioned Alma Economics to consider the key findings and recommendations from previous research ('Defining, measuring and monitoring democratic health in Wales' Valgardsson et al, 2023) exploring how democratic health is defined, measured, and monitored around the world, with a specific focus on how this learning could be applied in Wales. Alma Economics’ research study identified the options that could be taken forward to develop a specific set of measures and indicators to track, monitor, and assess different components of democracy in Wales.

Methodology

The study was conducted in three consecutive stages, as outlined below.

Stage 1 involved a desk-based review of academic and grey literature (e.g. policy documents) to enhance understanding of the key issues, ground the development of subsequent research materials, and identify areas of further enquiry. The review focused on existing evidence on how democratic health is currently measured in Wales, as well as on comparisons of the measures used internationally. Surveys and datasets available in Wales and reports on election results were also reviewed. A long list of indicators was created as a result of this, categorised according to the six dimensions of democratic health outlined in the WCPP report ‘Defining, Measuring, and Monitoring Democratic Health in Wales’ (Valgardsson et al, 2023): i) electoral democracy, ii) participatory democracy, iii) deliberative democracy, iv) egalitarian democracy, v) institutional responsiveness, and vi) transparency and media freedom. The indicators were further split into two groups: those indicators already collected in Wales, and indicators available elsewhere (in the UK or internationally) but not yet measured in Wales. Based on the evidence review, a bespoke Quality Assessment Framework was also created to evaluate the identified indicators in later stages of the research.

Stage 2 included conducting 2 online workshops to draw on key stakeholders’ knowledge and expertise to feed into the design of the Delphi study in Stage 3. These lasted up to 90 minutes and began with a brief presentation on current measures of democratic health as identified in Stage 1, followed by a discussion on the long list of indicators and the Quality Assessment Framework. Eight stakeholders fed into the design of the Delphi study and the finalisation of the Quality Assessment Framework through these workshops.

Stage 3 involved the Delphi study ('How to Conduct a Delphi Study' Chuenjitwongsa, 2017), consisting of three consecutive questionnaires that were designed to identify where there is consensus amongst a diverse panel of experts on indicators best placed to measure democratic health. Each questionnaire was available for completion for 2 to 4 weeks. To form the Expert Panel, a stakeholder mapping exercise took place to identify experts who: i) had familiarity with Welsh democratic health or a comparable context, ii) had applied/worked with similar measurements in the past (e.g., Freedom House, V-Dem, etc.), iii) had at least 3 years of experience and an established professional reputation in professions such as academia, policy, civil society, or electoral commissioning, and iv) were English or Welsh-speaking. Through this approach, an Expert Panel of 23 participants from Wales, the wider UK, and international settings was formed. Given the demanding nature of Delphi participation, the Expert Panel was encouraged to complete all rounds of the Delphi study, but participants were not penalised for missing a round and were allowed to complete subsequent ones.

The first round of the Delphi study focused on the initial shortlisting of indicators identified through the desk-based review and refined during the expert workshops, engaging 22 participants. The second round focused on evaluating the indicators shortlisted in Round 1 against the Quality Assessment Framework, asking experts to indicate their level of agreement with a series of quality assessment criteria per indicator. Due to being highly technical, the second round of the Delphi study was split into 2 parts. In Part A, 16 experts contributed, and in Part B, 10 experts contributed. The third and last round of the Delphi study focused on gathering feedback on the final short list of indicators proposed to measure and monitor democratic health in Wales, as these had been collated based on the results of round 2, confirming consensus on which indicators are most relevant, feasible, and reliable for use in the Welsh context. This final round engaged 21 participants.

The Delphi study resulted in a set of 6 indicators considered effective and relevant for measuring and monitoring democratic health in Wales.

1. Electoral democracy: Do voters have meaningful choices between candidates? This indicator is not currently available in Wales. It requires expert analysis to assess whether voters had genuine options to choose from in an election. V-Dem measures this through its ‘Elections multiparty’ variable, while Perceptions of Electoral Integrity evaluates whether voters were offered a real choice at the ballot box. It achieved 67% consensus among experts in the third round of the Delphi study.
2. Participatory democracy: Voter turnout in General, Senedd, and Local elections. This indicator is currently available in Wales. It measures the share of eligible voters who actually cast a ballot, based on official records of general elections (House of Commons Library), Senedd elections (Welsh Parliament), and local elections (The Electoral Commission). It achieved 81% consensus among experts in the third round of the Delphi study.
3. Participatory democracy: “Have you contacted your local councillor in the past 12 months, for example, with an enquiry, complaint, or problem?”. This indicator is currently available in Wales. Data points available for this indicator were collected through the National Survey for Wales between 2018 and 2022. It achieved 71% consensus among experts in the third round of the Delphi study.
4. Egalitarian democracy: Equality of political engagement & balanced demographic representation in government and legislators: demographic characteristics of elected candidates. This indicator is currently collected in Wales. Data points available for this indicator are collected via the Local Government Candidates Survey (subject to response rates) and the Senedd Elected Candidates Register (House of Commons Library) (though not self-reported but inferred from expert reporting). It achieved 71% consensus among experts in the third round of the Delphi study.
5. Institutional responsiveness: “On the whole, are you satisfied or dissatisfied with the way that democracy works in: a) the UK as a whole; b) Wales?”. This indicator is currently available in Wales, collected in waves as part of the Welsh Election Study. It achieved 81% consensus among experts in the third round of the Delphi study.
6. Transparency and Media Freedom: Can decision-makers be identified and held accountable for major policy and spending decisions? This indicator is not currently available in Wales. The approach applies the T-Index methodology for assessing transparency, drawing specifically on the Corruption Perceptions Index (Transparency International). The measure is based on 14 expert-coded questions. It achieved 71% consensus among experts in the third round of the Delphi study.

Through 3 rounds of expert engagement, the Delphi revealed expert consensus around a focused set of 6 indicators spanning 5 of the 6 democratic dimensions originally proposed: i) electoral democracy, ii) participatory democracy (2 indicators), iii) egalitarian democracy, iv) institutional responsiveness, and v) transparency and media freedom. Notably, no indicators from the deliberative democracy dimension met the shortlisting criteria, largely reflecting experts' concerns about the subjectivity inherent in assessing deliberative quality and their lack of confidence in the available expert-coded measurement approaches for all indicators in this dimension. The shortlisted set as a whole commanded 62% consensus among experts in the third round of the Delphi study.

Throughout the study, experts raised several considerations regarding data collection and methodological robustness. For indicators based on expert judgement, they emphasised the importance of clear conceptual definitions, transparent coding frameworks, and, where feasible, the use of techniques such as Item Response Theory (Cambridge University Press) to account for variation in experts’ scoring and to avoid overreliance on any single respondent’s assessment. For indicators derived from official statistics, experts highlighted their reliability and comparability over time, but also noted that such sources may overlook informal or emerging forms of democratic engagement and can be constrained by existing classifications and reporting practices. For survey-based indicators, they pointed to risks of social desirability bias, misreporting, and differences in relevance across groups, stressing the need for careful question wording and disaggregation by key demographic and geographic variables.

The indicators shortlisted under electoral democracy and transparency and media freedom are subject to expert analysis, and neither of them is currently available in Wales. It is understood that establishing 2 new indicators can be resource-intensive and time-demanding. To allow for measuring and monitoring democratic health in Wales in the meantime, 2 alternative indicators capturing similar aspects have been identified. However, these 2 alternatives come with caveats, as these indicators had not been shortlisted.

The indicator measuring meaningful choices between candidates could be replaced with another indicator from the same dimension, albeit capturing electoral integrity and not competitiveness: “How confident, if at all, are you that you know how to go about registering to vote?” (currently measured in Wales through the Public Attitudes Survey (The Electoral Commission)). There was no alternative from within the transparency and media freedom dimension to replace the indicator on accountability for major policy and spending decisions, so the second alternative is drawn from the institutional responsiveness dimension, capturing citizens’ satisfaction with government and the political system: “To what extent do you think your local council(s) act(s) on the concerns of local residents?” (currently measured in Wales through the National Resident Survey (Data Unit Wales)). Taking on board these 2 replacements further reduces the dimensions represented within the final set of 6 indicators from 5 to 4. Thus, the 2 alternatives are only suggested as a temporary solution until the 2 expert analysis indicators are ready to be measured for Wales.

Report authors: Alma Economics

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Public Services and Local Government
Knowledge and Analytical Services
Welsh Government
Cathays Park
Cardiff
CF10 3NQ

Email: Research.PublicServices@gov.wales

Social research number: 17/2026
Digital ISBN: 978-1-83715-979-6