Chapter 6 Comments and Final Remarks

In this study, the multi-criteria approach is used to obtain rankings for a number of Italian universities, according to certain curricular and job placement graduate characteristics.

The Electre model and its variants are well-known methods that help decision­makers in choosing a solution among a set of possible solutions reflecting their preference structure. In our case, the preference structure is provided (or produced) by the different scenarios. In general, no decision that is simultaneously the best choice from all the points of view considered as being relevant for dealing with the decision-making problem. Instead, there is ensemble set of solutions, generally numerous, that provide a logical framework for the choice of a “compromise” solution between the problems and the values inspiring the evaluator.

The application of a multi-criteria model should be followed by a sensitivity analysis of the final hierarchy of alternative solutions, in relation to the preference, indifference and veto thresholds, as well as the importance coefficients [91]. Sensi­tivity analysis was performed by changing the weights assigned to the criteria and including or excluding the key criteria: overeducation and mismatching (following the definitions of [80, 79]).

Generally speaking, as underlined in the previous paragraph, the results show many changes in the different hypothesized scenarios and in the observed periods.

In the north and centre of Italy, high flexibility in the labour market, even if lower during the economic crisis, accounts for the high ranking of universities in this region.

The Electre model stresses this situation very well. In fact, university α₇ is usually first (it is in first position in graduate employment rates; see the Almalaurea and Stella annual reports). In the same way, universities located in regions where the job market is rigid, thus offering few job opportunities to young people and graduates in particular, occupy the last positions of the ranking.

The flexibility of the model allows an analysis of the different performance of the universities in the middle position of the rankings. The scenario and, above all, the emphasis placed on the two key criteria can produce deep changes in the rankings of the universities. For example, university α₁ suffers from the economic crisis; it occupies the middle positions in all rankings in the pre-crisis period and is often in last position during the crisis (Fig. 5.1). Considering the key criteria, α₁ is more sensitive to mismatching than overeducation. University a6 is in the lower part of the rankings. Its position worsens when the J criteria are emphasized in the pre-crisis period (Fig. 5.1, models J1, J2 and J3) and during the crisis period when only mismatching is included (Fig. 5.1, model J6). It appears that university reputation produces the worst results. The key criteria do not change the rankings of a6. During the crisis, the position of a2 in the rankings improves, except when university reputation is emphasized (Fig. 5.1, models U4, U5 and U6 vs. J4, J5 and J 6).

Moreover, according to several scholars [3,8,16], overeducation and mismatching depend on the field of study. This means that their contribution to the university ranking may change accordingly.

In fact, if the ES graduates subset is considered, overeducation causes a loss in position for universities a6 and a1 before the economic crisis, and for universities a1 and α₅ during the crisis. Mismatching causes a loss in position for universities α₄ and a3 before the economic crisis, in scenario J. During the crisis, mismatching causes a loss in the position of universities α₆ and α₅.

If PSS graduates are considered, overeducation causes a loss in the position of universities α6 and α5 before the economic crisis, and for universities α1 and α5 during the crisis period. Mismatching causes a loss in the position of α₁, in Scenario U before the economic crisis, and for universities α₁ and α₅ during the crisis period.

The results and comments summarized above show that the Electre III method seems appropriate for such an analysis.

This methodology combines high flexibility with the subjective components pro­vided by the choices of decision-makers and the robustness of the results (e.g. the first and last positions are stable, and only few changes occur). Moreover, Electre III models compute the outrank relationship for each pair of alternatives and for each criterion. It is able to capture the complexity of the data (criteria and alternatives) because the outrank relationship shows whether one alternative is preferable or indif­ferent to another one, according to certain initial conditions (thresholds and weights). These methods rank the alternatives considering them together as covariates. Thus, the method is based on the individuation of the relationships between pairs of criteria that outrank or are outranked by other criteria.

The classic methods used for ranking statistical units, even if weights, standard­ization methods, and aggregation functions change, give only the best and unique ranking.

In this paper, the stability of the configuration of some alternatives (a₇, a₅ and a₆) at the top or in the last position of the rankings, and the different positions for other alternatives, according to the scenario, criteria and data sets, suggest that the subjec­tive elements introduced in the model are not that far from reality. Overeducation and mismatching are more evident in the 2011 rather than in the 2006 survey (Tables 4.2,

4.3 and 4.4). The key criteria rates are far higher than the evidence available in the international literature (see, e.g., [7], par. 2) and national literature (see, e.g., par. 2.3; [37, 9]). The obtained rankings confirm this key criteria role.

Obviously, this method is influenced by the subjective elements.

According to the interests of a specific stakeholder, weights and thresholds may be defined differently. Thus, the ranking of alternatives depends on the values of the thresholds and weights adopted.

These are the weakness and strengths of this approach. To reduce the intensity of weakness, many authors are trying to implement methods and rules for a better choice of thresholds and weights [94].

From a substantive point of view, by applying this flexible approach, we found that the rankings published every year by many institutions are not the unique con­figuration of the real state-of-the-art of the universities. By changing the method, the results can be surprising, showing, for example, that some small universities or uni­versities in the south of the country could perform better than the big universities in the northern regions. Moreover, in previous papers, this methodology has been used considering subjects as alternatives [1, 5], but they did not consider overeducation or mismatching. So the results are not comparable.