Composite indicators of scientific research: the robustness of university rankings based on composite measures

Composite indicators (CIs) which compare institution (e.g. schools, universities etc.) performances are useful tools in policy analysis and public communication. However, CI development involves stages where a number of subjective judgements have to be made, from the selection of individual indicators, to the choice of normalisation methods, weighting schemes, aggregation models etc. All these choices are potential sources of uncertainty which should be addressed when constructing a CI because they can have a significant impact on the composite index and rankings of the individual units considered. In this work - after a revision of the main university research evaluation methods and an illustration of the performance benchmarking system implemented in Italy - we address some of the methodological challenges implied in the construction of CIs and investigate the degree to which composite measures are an appropriate metric for evaluating and ranking the research performance of Italian universities.