TY - JOUR
T1 - Multiple-criteria performance ranking based on profile distributions
T2 - An application to university research evaluations
AU - Kunsch, Pierre
AU - Ishizaka, A.
PY - 2018/12
Y1 - 2018/12
N2 - This article addresses a category of multi-criteria ranking problems in which performance evaluations of the ‘objects’ or ‘alternatives’ to be ranked are not given by unique numbers but by performance profile distributions running over an ordered set of performance score levels. The numerical values to be given to score levels are not specified a priori using cardinal scales. A weighted sum approach is developed based on order statistics to combine the individual profile distributions. In this way, a global ranking indicator is obtained, considering not only mean distribution values but also standard deviations. As a test of feasibility, the resulting Profile Ranking with Order Statistics Evaluations (PROSE) approach has been applied to the performance profile distributions provided by the UK Research Excellence Framework 2014, evaluating the quality of UK research.
AB - This article addresses a category of multi-criteria ranking problems in which performance evaluations of the ‘objects’ or ‘alternatives’ to be ranked are not given by unique numbers but by performance profile distributions running over an ordered set of performance score levels. The numerical values to be given to score levels are not specified a priori using cardinal scales. A weighted sum approach is developed based on order statistics to combine the individual profile distributions. In this way, a global ranking indicator is obtained, considering not only mean distribution values but also standard deviations. As a test of feasibility, the resulting Profile Ranking with Order Statistics Evaluations (PROSE) approach has been applied to the performance profile distributions provided by the UK Research Excellence Framework 2014, evaluating the quality of UK research.
KW - 2014 Research Excellence Framework
KW - Multi-criteria ranking
KW - Order statistics
KW - Performance profiles
KW - Ranking indicator
UR - http://www.scopus.com/inward/record.url?scp=85048960748&partnerID=8YFLogxK
U2 - 10.1016/j.matcom.2018.05.021
DO - 10.1016/j.matcom.2018.05.021
M3 - Article
SN - 0378-4754
VL - 154
SP - 48
EP - 64
JO - Mathematics and Computers in Simulation
JF - Mathematics and Computers in Simulation
ER -