- ERA conference blog - https://blogs.ec.europa.eu/ERAconference09 -

Which indicators to measure progress? Comment by José Ignacio Alonso

José Ignacio Alonso

 Discussant session 1.8

 

1. Introduction

Science, technology and innovation systems are complex and extremely heterogeneous. Thus, its monitoring and governance are a grand challenge. In this context, the objective is to monitor the building process of European Research Area (ERA) and that it becomes the motor of a competitive Europe in a globalised world. However, the need to monitor this process passing through the definition of a set of indicators and find a model or framework that allows a representation of reality to measure. Therefore:

a)         Indicators that promote the development of evidence- based monitoring systems on progress towards the ERA (European Research Area) and knowledge –based society.

b)         Should propose a model structure or simplified representation enabling a view of reality to measure and where may be easy to fit the set of indicators defined.

 

On the basis of these premises, the proposal of “the ERA indicators Framework”, on the Report of the Expert Group, “ERA Indicators and ERA Monitoring”, is an excellent proposal towards the desired objective

However, the model or “ERA indicators Framework” report raises some questions. According to the report the proposed model is a “no well-specified model”, where the causes and effects are ambiguous. These ambiguities can be converted into shortcomings, therefore, be analysed below.

 

2.           COMMENTS ABOUT THE CONCEPTUAL FRAMEWORK

2.1.       GENERAL COMMENTS

The proposed model of the above mentioned report is divided in two structural dimensions: the “components” of the ERA and the “types of concern”. Nevertheless, as pointed in the document, there are some difficulties to interpret the conceptual framework due to the difficulty of distinguishing between components and types. However, we consider it necessary to make  an  effort to make more understandable this difference and  improve any deficiency of it for using it  as a powerful tool for analyse and diagnosis of the ERA,   and as an instrument to guide the actions that can be undertaken. In general terms, we think it is an excellent proposal to achieve the objective desired.

Analysing the proposed model, we consider one of the axis or dimensions proposed, in particular, the axis “the four” types of concerning for the ERA monitoring, (even though there are not specifications of the causal relations among every individual element and they may be considered as a weakness), the difference between the four types of concern which relate to input (National or EU-level policy actions) –intermediate outputs (ERA progress) – final outputs (knowledge society – Lisbon objectives) is correct. We prefer, in order to provide a clear picture of this classification by types, that is named type B as ERA Progress (or state) instead of ERA Making.

However, the other dimension structural or axis of the model, “the components of the ERA”, which can more objected, and perhaps consider seeking or defining other “components”. A definition of the proposal components or any other proposal arises by the difficulty to identify and classify the more essential aspects of ERA, which are coming from the general report “European Research Area Vision 2020.

In particular, on the proposal for a model especially emphasizes the social dimension in the society- science dialogue and its relevance on the sustainable development and social welfare since they are not plainly summarized on the before mentioned report . We refer to components 4 and 5. The inclusion of these aspects can be a handicap due to the need of defining new indicators, and to the difficulty and concretion of its assessment.

 

2.2.       SPECIFIC COMMENTS ABOUT COMPONENTS

In addition, it would be necessary to define in a more precise way the different components especially the first three (1.- Knowledge Activities: Volume and Quality, 2.- Knowledge Triangle: Flows and dynamics y 3.-Fifth freedom: intra and extra-EU openness and circulation). Thus may prevent ambiguities and overlaps In order to avoid them, it would be necessary to establish clear boundaries.

For example, between components 1 and 2 (1.-Knowledge Activities: Volume and Quality, 2.- Knowledge Triangle: Flows and dynamics), it can be  stated that:

          The quality of Knowledge Activities might include some aspect of the “Knowledge Triangle” which component 2 is based (coordination of policies, and to strength the innovation and investment, cooperation between private and public sectors, etc)

          Some activities, which are measured transversally in volume and quantity among the MS of EU (component 1), would be showing already provided aspects of the component 2 as well.

With relation to possible overlays between components 2 and 3 (2.-Knowledge Triangle: Flows and dynamics y 3.- Fifth freedom: intra and extra-EU openness and circulation) it can be highlighted the difficulty to establish a clear boundary  among interactions or flows (component 2)  and researchers, knowledge and technology  mobility (component 3) and particularly:

          The double national dimension (within MS) and trans-national (among MS of the UE) of this component can be included in component 3.

          Component 3 includes interrelations between MS within every pole of the knowledge triangle (innovation, research and education) that could be considered included in the flows among MS of component 2.

The following comments come from analysing individually the different components:

          It would be necessary to summarize the interrelation between private and public sectors within the “knowledge triangle” in component 2.

          Component 5  (Sustainable development and Grand challenges) must include a set of comprehensive indicators to provide information about sustainable development (economic, energetic, production areas, new labour processes because of the inclusion of TIC’s, balance among personal family and professional life ,  integration for people with disabilities,  continuous training of Human Resources etc).

According to the above mentioned comments, it is obvious the difficulty to define system components of this structural axis.  However, in order to state the validity of the proposed model, we think that is necessary to analyse and consider it for a better concretion and so that, an improvement of the proposed model.

We think that the designed framework design as well as the analysis of the proposed indicators demands a systematic effort as well as a rigorous work that should be closely analysed as it is one of the major contributions of this report.

The proposed model emphasizes the “social” dimension and the society-science dialog and its relevance on the development and social welfare. In this regard, and taking into account the necessity to consider the role that science plays on the improvement of the  citizen quality of life,  there are not clear boundaries  between components 4 “The Societal Dimension” and 5 “Sustainable development and Grand challenges”.

 

3.           comments about the proposed indicators

A great effort has been made trying to integrate and quantify the different indicators on the double dimension of the proposed model on the document. However, as the authors already have mentioned, this integration does not avoid the absence of indicators in some cases when making the intersection between rows (types of concern) and columns (components) of the proposed model.

After reading the working document and analysing the different sets of proposed indicators of the model, we would like to suggest more disaggregated indicators to provide more information to measure the ERA progress. Some indicators are already known and it is possible to compile data for its generation, and others have been proposed by EUROSTAT, and they are underdeveloped.

Some of these mentioned indicators are the following:

3.1.       Public investments in knowledge [Type A1, Component 1]

In order to provide more information it can be showed indicators with disaggregated data and providing a more detailed information on:

          Public Investment in RD[1] Human Resources (obtaining part  of the GOVERD budget used to support  training, grants, mobility etc in Human Resources) [also Type C, Components 1,2 & 3]

          Public investment to support private sector[2] (share  of the  GOVERD devoted to promote RD private sector and its comparison with BERD[3]) to RD  Public Administration (share of  GOVERD devoted to RD Public Administration )

 

3.2.       European Integration of Research System (policies) [Type A2, Component 1]

3.2.1.       MS Participation on the Framework Programs

At the heart of the Lisbon Strategy, research is a component of a knowledge triangle (the other two being education and innovation) meant to boost growth and employment in the European Union (EU) in the context of a global economy. The current 7th Framework Program for Research (2007 to 2013) and previous programmes are and they have been very important an opportunity for the EU to match its research policy to its ambitions in terms of economic and social policy by consolidating the European Research Area (ERA).

It would be interesting to know to what extent MS participated on the several Framework Programs and to know their contributions as well as their returns. Returns values could be divided by participant entity and thematic areas.

3.2.2.       National Public Funding to Trans-Nationally Coordinated Research[4]

Trans-national cooperation in research is one of the core elements of the ERA and developing relevant indicators for its measurement is very important for policy based debates on public funding. In order to improve the policies for the development of ERA it is necessary to know what amount of public funding of research in the ERA is directed to trans-national cooperation.

Evidence based information is needed in particular on the trans-national, European dimension of national R&D policies. The fundamental need is distinction to be made between public funds for R&D activities that are spent purely nationally and public budgets that are spent in a coordinated or collaborative way between countries within the ERA. This requires further development of appropriate new indicators and investigation of the feasibility for data compilation on them.

We can then include a set of indicators that can be measured this type of funding

1.       Share of national public funding to trans-nationally coordinated research[5] – defined as national public funding to coordinated research as a percentage of total public funding of R&D (GBAORD) in a given country.

2.       Share of MSs’ public funding to coordinated research at EU level [6]defined as a sum of MSs’ public funding to coordinated research as a percentage of total public funding to coordinated research at EU level (MSs’ + Community budget funding allocated for coordinated research).

3.       Share of national public funding to trans-national public R&D performers – defined as national public funding to trans-national public R&D performers as a percentage of total national public funding to coordinated research.

4.       Share of national public funding to Europe-wide trans-national public R&D programs – defined as national public funding to Europe-wide trans-national public R&D programs as a percentage of total national public funding to coordinated research.

5.       Share of national public funding directed to bi- or multi-lateral public R&D programs established between MSs governments – defined as national public funding to bi- or multi-lateral public R&D programs as a percentage of total national public funding to coordinated research.

The indicators 3, 4 and 5 measure the importance of different types of trans-national public R&D actions in national public funding to trans-nationally coordinated research.

3.2.3.       European Regional Development Fund (ERDF) efficiency indicators [Types A1 & A2, Components 1-5]

The aim of the innovative measures to which the ERDF contributes is to reinforce competitiveness in Europe by reducing the gaps between regions and supporting innovation, RTD and the use of new information and communication technologies[7].

It is necessary to monitor the innovative measures must therefore concentrate on three priorities: a) regional economy based on knowledge and technological innovation; b) the information society at the service of regional development (e Europe-region) and c) regional identity and sustainable development. And on the other hand, implementing the innovative measures should also make it possible to: 1) improve the quality of assistance under the Objective 1 and 2 programs to which the ERDF contributes; 2) enhance and strengthen the public-private partnership; 3) exploit the synergies between regional policy and the other Community policies and 4) have exchanges between regions and collective learning by means of the comparison and spread of best practice.

In this context the question about developing a relevant set of indicators that may be particularly pertinent due to the policy needs for tracking the use and ensuring more effective deployment of the Structural Funds for R&D.

Next indicators have been proposed by EUROSTAT:

1.       Total amount of Structural Funds for R&D at national level is defined as a sum of Community budget received from this particular EU funding source (SF for R&D) plus the cofinancing amount of national government budget (state, federal, provincial), as measured by GBAORD, directed to R&D projects financed by the EU Structural Funds.

This indicator comprises two sub-categories:

             Community budgetary money allocated to the MSs through Structural Funds for R&D

             National co-financing amount of public funding of R&D projects financed by the EU Structural Funds

2.       Share of national public funding to EU Structural Funds – defined as national public co-fundingof R&D projects financed by the EU Structural Funds as a percentage of total public funding of R&D (GBAORD) in a given country.

3.       Share of Structural Funds for R&D at EU level – defined as Community budgetary money allocated to the MSs through Structural Funds for R&D as a percentage of total Community budget.

4.       Proportion of national public co-funding part vs. Community budget part in R&D projects financed by the EU Structural Funds.

 

4.           CONCLUSIONS

The developed model for monitoring ERA as well as the analysis of the proposed model,    its systematization and involvement of the proposed model is a rigorous and  excellent work that it should be analysed in detail. The proposed comments and suggestions are a small contribution to improve the performance of ERA.


[1]  Table 4. Tertiary education expenditures as % of GDP.

[2] Table 4. Assessment of public financial support and tax incentives for private R&D.

[3] Table 5. Publicly financed BERD as % of publicly financed R&D. Share of BERD financed by government.

[4] In the Table 4 appears this new set of indicators.

[5] This indicator measures the extent to which national research policy is directed towards funding of European coordinated research.

[6] This indicator measures the proportion of MSs’ public investments versus Community budget funding for coordinated research.

[7] It therefore forms part of the strategy approved at the European Council in Lisbon on 23/24 March 2000, which aims at boosting employment, economic competitiveness and social cohesion in the framework of a knowledge-based economy.