EVALUATION OF THE INNOVATION LEVEL IN THE EU COUNTRIES IN 2016 AND 2011

Purpose – The main aim of this paper is to evaluate the innovation level in the European Union countries, but in a different way than it was adopted in the European Union reports (European Innovation Scoreboard, and earlier Innovation Union Scoreboard). Analyses were performed in relation to INPUTS-OUTPUTS. Research method – Two methods of multi-criteria analysis were used in the calculations. The first is the popular method known as Simple Additive Weighting. The second is the method known as Processing Technique of Ratings for Ranking of Alternatives, which was developed by the author of this article. Results – The obtained results of the innovation level were analyzed. Analysis concerned two periods: 2016 and 2011 and determined aggregate ratings, which characterized the innovation level of particular European Union countries. Four classes of the innovation level were defined: innovation leaders, good innovators, week innovators and innovation outsiders. Then, based on the calculated global ratings, European Union countries have been assigned to the appropriate classes. The results of this analysis were compared with the classification of the EU Member States in terms of the innovation level that IUS / EIS reports contain. Analyses in relation to INPUTS-OUTPUTS have also made it possible to assess the usefulness of indicators from the IUS / EIS reports to measure the innovation level in such a way. Originality /value / implications /recommendations – The analysis of the innovation level was carried out using two multi-criteria analysis methods.


Introduction
The current socio-economic challenges are forcing economies and entities functioning in them to be constantly innovative. Innovation is ubiquitous, and generally speaking, is the result of interaction of internal factors, such as the expenditure incurred by national operators on R&D activities or investments in the training of human resources. Economic development is possible in the presence of the trained human capital. With properly trained staff to increase the innovative capacity of businesses, possibilities of economic development are greater [Silverberg et al., 1994, p. 21]. Furthermore, innovation also means a new combination of existing opportunities and is characterized by a highly important function of the development, mainly in relation to the economic condition [Fagerberg et al., 2005, p. 6]. Various aspects of innovation and its importance for economic development have been recently discussed in scientific works [Hall, Rosenberg, 2010a, p. 804;2010b, p. 600].
When aiming to strengthen the socio-economic growth and accelerate the modernization of the European Union industry, the uptake of product and service innovations, the use of innovative manufacturing technologies and the introduction of new business models are necessary. The European Commission develops policies that help to speed up the broad commercialization of innovation and engages in many activities that support innovation in the EU. The European Commission provides various tools that map, monitor and assess the EU's performance in different innovation areas. The provided information helps policy makers and practitioners at the EU, national and regional levels to benchmark their performance and policies, to identify good practices to follow and to learn about new trends and emerging business opportunities that can inform evidence-based policy making.
One of the European Commission's tools for monitoring the innovation level is the European Innovation Scoreboard (EIS). The European Innovation Scoreboard -previously the Innovation Union Scoreboard (IUS) -provides a comparative analysis of innovation performance in the European Union member states. It assesses the relative strengths and weaknesses of national research and innovation systems and helps countries and regions to identify the areas they need to address. The European Innovation Scoreboard is important for each country to monitor their own innovation level, as well as to compare this state with the achievements of other countries.
So far several reports of the European Innovation Scoreboard (2007, 2008, 2009, 2016, 2017, 2018and 2019) and the Innovation Union Scoreboard (2011, 2013, 2014, 2015and 2016 have been published [www1]. These reports have been based on appropriate sets of innovation indicators, which include several highlighted areas of innovation. It can be added that the set of indicators in the EIS / IUS reports is modified from time to time. This is to achieve the greatest usefulness of these reports tin terms of proper diagnosis of particular aspects of innovation. In this context it is, however, important to point out, that the reports of the Innovation Union Scoreboard [IUS, 2011[IUS, , 2013[IUS, , 2014[IUS, , 2015 and the European Innovation Scoreboard [EIS, 2016] consistently operated the same set of indicators for several previous years. Therefore, the article uses data from two reports: IUS [2011] and EIS [2016].
All reports of the European Union Scoreboard contain a detailed analysis of the level of innovation in the EU member states. Of course, it is possible to prepare a different approach to the analysis of relevant data on the innovation level in the EU countries, which will give a broader look at the analyzed problem. Therefore, this article presents a different approach, which is based on the use of the chosen methods of multi-criteria analysis. Another difference from the approach used in the IUS / EIS reports is the consideration of innovativeness in terms of INPUTS-OUTPUTS, which will be explained in the next section.

Characteristics of the analyzed data
This paper will analyze innovation indicators, which are included and presented in the two reports: the report at the beginning of this decade [IUS, 2011] and the last report with the same set of indicators [EIS, 2016]. In accordance with the approach taken by the authors of these reports, the adopted innovation indicators are assigned to three groups: Enablers, Firm activities and Outputs. The Enablers group contains the main drivers of innovation performance external to the firm and differentiates between 3 innovation dimensions: 1) Human resources -includes 3 indicators and measures the availability of highly skilled and educated workforce (New doctorate graduates, Population aged 30-34 with completed tertiary education, Population aged 20-24 with upper secondary level education).

2) Open, excellent and attractive research systems -includes 3 indicators and measures
the international competitiveness of the science base (International scientific co-publications, Scientific publications among top 10% most cited, Non-EU doctorate students). 3) Finance and support -includes 2 indicators and measures the availability of finance for innovation projects by venture capital investments and the support of governments for research and innovation activities (R&D expenditure in the public sector, Venture capital investments). The Firm activities group comprises the innovation efforts at the level of the firm and differentiates between 3 innovation dimensions: -Firm investments -includes 2 indicators of both R&D and Non-R&D investments that firms make in order to generate innovations (R&D expenditure in the business sector, Non-R&D innovation expenditure). - Linkages & entrepreneurship -includes 3 indicators measuring innovation capabilities (SMEs in-house innovation, Innovative SMEs collaborating with others, Public-private co-publications). - Intellectual assets -comprises different forms of Intellectual Property Rights (IPR) generated as performance in the innovation process (PCT patent applications, PCT patent applications in societal challenges, Community trademarks, Community designs). The Outputs group presents the effects of firms' innovation activities and differentiates between 2 innovation dimensions: -Innovators -includes 3 indicators measuring the share of firms that have introduced innovations onto the market or within their organizations, covering both technological and non-technological innovations and employment in fast-growing firms of innovative sectors (SMEs introducing product or process innovations, SMEs introducing marketing/organizational innovations, fast-growing innovative firms). -Economic effects -includes 5 indicators and presents the economic success of innovation (employment in knowledge-intensive activities, medium and high-tech product exports, knowledge-intensive services exports, sales of new-to-market and new-to-firm innovations, license and patent revenues from abroad). A detailed description of these innovation indicators is shown in table 1. In the approach adopted by the author of this article, the groups of Enablers and Firm activities can be regarded as INPUTS, which are the determinants of the innovations level, which is determined by the OUTPUTS. Further analysis includes creating rankings of the European Union member states in the innovations field (in terms of Inputs and Outputs). The analysis was carried out basing on the data that characterize the current level of innovation in the EU member states (2016 year). For comparison, the state of innovations in the year 2011 will also be analyzed.

Ranking creating using methods of multi-criteria analysis
Multi-criteria analysis (MCA) represents a body of techniques potentially capable of improving the transparency, auditability and analytic rigour of taken decisions [Dunning et al., 2000, pp. 3, 7-14]. The MCA framework ranks or scores the performance of alternative decision options against multiple criteria which are typically measured in different units. MCA emerged as a decision analysis technique in the 1960s and 1970s, partly resulting from the rapid growth of operations research. Today MCA is an established methodology with dozens of books, thousands of applications, dedicated scientific journals, software packages and university courses [Figueira et al., 2005[Figueira et al., , p. 1045. MCA can be defined as a decision model which contains: a set of decision options which need to be ranked or scored by the decision maker; -a set of criteria, typically measured in different units; -a set of performance measures, which are the raw scores for each decision option against each criterion. The MCA model is represented by an evaluation matrix D of m decision options in the light of n criteria. The initial performance score for decision option i = 1,2,..., m with respect to criterion j = 1,2,..., n is denoted by Di,j. A minimum requirement for the MCA model is at least two criteria and two decision options (m ≥ 2 and n ≥ 2). The importance of each criterion is usually given in a one dimensional weights vector w containing n weights, where wj denotes the weight assigned to the j-th criterion. It is possible for D and w to contain a mix of qualitative (ordinal) and quantitative (cardinal) data.
Later in the article, two techniques of MCA will be used in the creation of the ranking of the EU member states. The first is a so-called Simple Additive Weighting (SAW) method [Tofallis, 2014, pp. 109-119]. The second of the used methods is the original author's proposal that was presented as a PROcessing TEchnique of Ratings for Ranking of Alternatives (PROTERRA) [Kobryń, Prystrom, 2018, pp. 1-14]. In the analyses, the EU countries were adopted as decision options, while innovation indicators were used as criteria.

Analysis of the innovation level in the European Union countries
The assessment of the innovation level in the EU member states focuses on the latest data, taken from a report of the European Innovation Scoreboard [EIS, 2016], which is a part of the European Innovation Scoreboards project. Additionally, in order to compare the current state with an earlier state, the data from the report of the Innovation Union Scoreboard [IUS, 2011] were also analyzed.
In the light of the analysis purpose, which was to determine the aggregated ratings of the innovation level of the individual European Union countries, all indicators (as evaluation criteria) were treated as equally important, giving them equal weight values. The analysis was performed separately for INPUTS (Enablers and Firm activities) and OUTPUTS. The calculations results are presented in the following tables :  -table 2 shows the aggregated ratings for INPUTS and OUTPUTS obtained  by the weighted summation method,  -table 3 shows the aggregated ratings for INPUTS and OUTPUTS obtained by the PROTERRA method. At the bottom of the tables 2 and 3 there are also given statistical measures, which were used in subsequent analyzes. The following generalized designations were adopted: i Q -the aggregated rating of i-th object (option) obtained by use the SAW method or PROTERRA method, Q -an average value of all aggregated ratings obtained by an appropriate method, s Q -standard deviation of Q .
On the basis of the ratings shown in tables 2 and 3, the EU countries were classified into four groups: innovation leaders, good innovators, weak innovators and innovation outsiders. They were based on the following classification criteria: -innovation leaders, when:  Assignment of particular countries to the above classes is illustrated by tables 4 and 5. The results shown in tables 4 and 5 allow to conclude that: a) In many cases, the methods SAW and PROTERRA provided consistent assignment of the analyzed countries to a particular class of innovation level.
In the light of data from 2016, there are 20 such cases in INPUTS and 17 cases in OUTPUTS. Whereas for data from 2011 there are 14 cases in INPUTS and 16 cases in OUTPUTS. b) In some cases, the methods SAW and PROTERRA provided a slightly different assignment of the considered countries to a specific class of the innovation level. It may be assumed that this is the result, among other things, of the fact that the PROTERRA method reflects a disparity the values of the initial assessments for particular objects/options to a greater degree [as signaled by: Kobryń, Prystrom, 2018]. c) When comparing sets of countries assigned to each class level of innovation in the INPUTS and OUTPUTS, a proper correlation is not always visible. This means that a certain level of innovation in the INPUTS is not reflected in a similar innovation level in terms of OUTPUTS. In the light of data from 2016 such correlation occurs in 10 cases analyzed by the SAW method and in 11 cases analyzed by the PROTERRA method. Whereas, in the light of data from 2011 such correlation occurs in 15 cases analyzed by the SAW method, as well as by the PROTERRA method. d) The lack of correlation in the assignment to the same class of the innovation level is not significant. They are usually placed in neighboring classes, which implies that the appropriate innovation level in the INPUTS generally translates to a similar innovation level in the OUTPUTS. This is confirmed by the correlation between aggregated scores for the

Good innovators Weak innovators Innovation outsiders
Belgium

Inputs Outputs Inputs Outputs
Innovation leaders

Good innovators Weak innovators Innovation outsiders
Attention: X -the same assignment to a specific class in terms of Inputs by the use of SAW method and PROTERRA method x -assignment to a specific class in terms of Inputs by the use of SAW method or PROTERRA method O -the same assignment to a specific class in terms of Outputs by the use of SAW method and PROTERRA method o -assignment to a specific class in terms of Outputs by the use of SAW method or PROTERRA method + -the same assignment to a given class in terms of Inputs and Outputs Source: author's own elaboration based on: [IUS, 2011;EIS, 2016].

Good innovators Weak innovators Innovation outsiders
Belgium Attention: X -the same assignment to a specific class in terms of Inputs by the use of SAW method and PROTERRA method x -assignment to a specific class in terms of Inputs by the use of SAW method or PROTERRA method O -the same assignment to a specific class in terms of Outputs by the of use SAW method and PROTERRA method o -assignment to a specific class in terms of Outputs by the use of SAW method or PROTERRA method + -the same assignment to a given class in terms of Inputs and Outputs Source: author's own elaboration based on: [IUS, 2011;EIS, 2016].
A separate issue are the changes in the aggregated ratings for 2016 compared to 2011. The results obtained by the PROTERRA method are illustrated in charts 1 and 2. In contrast, an illustration of the results obtained by the SAW method is shown in charts 3 and 4. The presented charts show significant differences in changes in the fields of INPUTS and OUTPUTS. There are relatively few cases where positive changes in the INPUTS translate into positive changes in the OUTPUTS. A similar state can also be noted in the case of negative changes, i.e. negative changes in the INPUTS are rarely accompanied by negative changes in the OUTPUTS.
In the opinion of the author of this article, this is a sign that the assessments for the OUTPUTS are not necessarily the result of the appropriate national innovation policy, which is measured by the innovation indicators included in the OUTPUTS group. This may indicate a certain imperfection of the indicators used by the European Innovation Scoreboard. Therefore, further work and research are recommended to improve the indicators set for measurement of the innovation level in the fields of INPUTS and OUTPUTS. Such an indicators set would allow national governments to create innovation policy more effectively and appropriately.

CHART 4 Changes in the aggregated ratings for OUTPUTS in 2016 compared to 2011 by SAW method
Source: author's own elaboration based on: [IUS, 2011;EIS, 2016].

Conclusions
Monitoring the level of innovation in the EU countries is an important issue and requires the use of indicators that are best suited to measure this very complex problem. On one hand, it is advisable to use the same set of indicators to compare the level of innovation in different periods. On the other hand, if it is justified, the need to continuously improve the available measures of innovation requires their modification. Such circumstances accompanied the analyses described in this article, limiting their time frames to the period 2011-2016.
The European Innovation Scoreboard (subsequent Innovation Union Scoreboard) is important for each country to monitor their own innovation level, as well as to compare this state with the achievements of the other countries. The innovation indicators, which were presented in the two reports: the report at the beginning of this decade [IUS, 2011] and the recent study [EIS, 2016], were analyzed. The analysis of the innovation level was carried out using two multicriteria analysis methods: the SAW method and the PROTERRA method, which is an original method proposed by the author of this article.
Based on the analysis it can be stated that: 1. in many cases, the SAW method and the PROTERRA method provided different rankings of the EU countries, which probably results from the fact, that the PROTERRA method allows a greater extent of maintaining the ratios between the values of the indicators and to take these proportions into account in determining aggregated ratings; 2. in 2016, in the innovation leaders group in the field of INPUTS we may include Denmark, Netherlands, Finland and Sweden, and in the field of OUTPUTS -Ireland and Luxemburg; 3. a slightly lower, but also a high innovation level in the field of OUTPUTS characterizes Denmark, Germany and Netherlands (note the high positions of Malta in the analysis by the PROTERRA method); 4. in 2016, in the innovation outsiders group in the field of INPUTS we classify Bulgaria and Romania (depending on the analysis method the outsiders are also Greece, Latvia, Poland and Slovakia) 5. and in the field of OUTPUTS -Latvia (depending on the analysis method the outsiders are also Bulgaria, Croatia, Cyprus, Lithuania, Poland and Romania). Based on the comparison of the results from the years 2011 and 2016, it can be stated that the present system of innovation indicators used by the European Innovation Scoreboard needs to be modified in order to better reflect the impact of input factors (namely Enablers, Firm activities) on the economic effects measured using output indicators.
In addition, it can be stated that the assignment of the individual EU countries to the appropriate group, which results from the use of the SAW and PROTERRA methods, shows high compatibility. The slight differences that can be noticed in individual cases are the result of different calculation mechanisms of these methods, which in turn affects the values of the synthetic scores. The aforementioned differences can be treated as a signal that this type of analysis and grouping of various objects compared with each other require the use of several methods to obtain more reliable results.