Session 1.8 conclusions: Which indicators to measure progress?
November 5, 2009The issue and context:
This parallel session addressed the question on the development of an evidence-based monitoring system on progress towards the ERA and a knowledge-based economy. This work forms part of the overall governance system of the Ljubljana process and contributes to the post 2010 EU strategy for Growth and Jobs.
The discussions were based on a report elaborated and presented by the expert group “ERA indicators and ERA monitoring”. The debate was further enriched through presentations by the discussants of the group and by the Chair of the session.
Discussion:
The expert group report is considered as a good basis for work on developing an evidence-based monitoring system of ERA and its contribution to a more knowledge-intensive economy in Europe. The concrete indicators proposed are very useful and relevant, but they constitute proxies that will be progressively improved.
Indicators:
It is important to specify the concept of ERA when setting up a monitoring system. Three complementary focus can be distinguished: a) ERA as the European internal market for research; b) ERA as a tool to achieve a knowledge-based society (Lisbon objectives); and c) ERA as a sum of structural components defined in the ERA Vision for 2020.
Indicators serve as a tool, a trigger to achieve better understanding but also to communicate, to mobilise, to ensure accountability of the main actors and to evaluate the impact of actions. The evidence-base has therefore to be elaborated in a simple and understandable way, but must still cover a comprehensive set of issues in order to represent the reality in a valid manner.
In this sense a comprehensive monitoring system should include “input” indicators, “system efficiency” indicators and “outcome and impact” indicators. In addition, there should be a systemic approach in the sense that no indicator should be seen independently. The session therefore welcomed the proposal of three corresponding sets of indicators. Some indicators proposed in the report of ERAB were also mentioned as useful (such as an indicator on the structural funds and on public procurement) as well as indicators on innovation (venture capital or an indicator on the growth of firms were mentioned).
Objectives:
A monitoring system does not consist only of indicators. Each indicator should be linked to a policy objective. There should be an effort to ensure 1 indicator for 1 objective, and if there would be two or more indicators for an objective, the indicators should be complementary, not alternatives. For the objectives of which the realisation implies a wide set of activities, a composite indicator constitute also an alternative to be considered.
The reconciliation between complexity and communication can indeed be enhanced by some composite indicators. This is also an instrument for “mobilising” the public and foster accountability. These composite indicators should be carefully designed both with respect to their individual components and for the weighting of each component.
Targets:
An important variable in the choice of policy targets is the issue of accountability: who is responsible for progress? Other aspects to consider on target-setting are the normative aspects of indicators (indicators are tools, not targets in themselves) and the reflection on what is the ideal situation (Is “more” always “better”?). The discussion stressed both the need for evidence-base and the realisation that the “optimal” level is mainly in the end a political decision.
One challenge is how to translate ERA targets at Member States level. The session discussed the need to make country grouping and the criteria to make meaningful groupings of Member States. Indicators that underline differences between countries are important in this context. When setting and interpreting progress towards targets it is important to take country-specific characteristics into account. Countries could be grouped based on countries with similar industrial structure, size, etc. The country grouping is also important to fully grasp the cohesion aspect of progress towards ERA. The taking into account the country specificity should avoid reducing the degree of ambition for each group of countries.
Conclusions/Recommendations:
1) The session broadly agreed on the structure proposed by the expert group report and on a majority of the proposed indicators.
2) An evidence-based monitoring system need to recon ciliate complexity with simple communicable policy messages, a more comprehensive understanding of reality with a clear accountability of the key actors.
3) A monitoring system needs to include both “input”, “throughputs” and “outputs and impacts” dimensions.
4) Further work is needed to develop composite indicators
5) The ERA monitoring system must also include measures of efficiency of ERA in fostering knowledge-based society controlling for different patterns of specialization. This concerns also the innovation gap.
6) For national target setting, specificities of countries must be taken into account without unduly reducing the degree of ambition.
7) The stakeholder consultations are key to ensure common understanding of the indicator and for ownership and mobilising effects of the targets.
Number of views: 766
Print This Post
Email This Post
RSS comments feed on this entry