Utility effectiveness, composite indicators and ‘benefit of the doubt’ models

How important are preferences when evaluating the performance of innovation systems?

Bibliographic Data

ID22412005
AuthorsJosé L Zofío (0000-0003-1170-9501, Erasmus MC Cancer Institute, corresponding author), Juan Aparicio (0000-0002-0867-0004, Universitat de Miguel Hernández d'Elx), Javier Barbero (0000-0002-2155-3996, Universidad Autónoma de Madrid), Jon Mikel Zabala‐iturriagagoitia (0000-0003-1975-2555, Universidad de Deusto)
Year2026
Volume106
Pages102528
Publication date2026-08-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueSocio-Economic Planning Sciences (JOURNAL)
Journal identifiersISSN: 0038-0121 • E-ISSN: 1873-6041
PublisherElsevier BV (PUBLISHER)
DOI10.1016/j.seps.2026.102528
OpenAlexW7164326689
LanguageEN
References cited55

This study introduces a methodology that embeds ‘Benefit of the Doubt’ evaluations of performance of innovation systems, based on objective measures of technical effectiveness—that compare the values of a set of indicators across countries—into the literature of composite indicators, which rank countries relying on utility functions and whose aggregating structure and weights are subjectively chosen by the decision makers. Considering the general case of the constant elasticity of substitution utility function, whose power mean formulation nests popular specifications such as the weighted (linear) arithmetic mean, weighted geometric (Cobb-Douglas) mean, and fixed-proportions (Leontief), we define the concept of utility effectiveness of an alternative as the ratio of its observed utility to the maximum utility observed across countries. We show that the utility loss resulting from an ineffective performance can be consistently decomposed into two mutually exclusive factors: first, grounded in the objective criterion of Pareto-dominance, the mentioned technical effectiveness is represented by a technical (radial) measure depending only on quantities, which gauges the distance to the utility frontier; second, a measure of allocative effectiveness, determining the additional utility loss resulting from an unbalanced composition of indicators given the preferences of the decision maker. Using data from the European Innovation Scoreboard, our results show high levels of utility ineffectiveness in Southern and Eastern European countries. On average, these countries’ performance is half of the maximum utility reached by the best performing innovation systems. Objective technical ineffectiveness generally dominates over subjective allocative ineffectiveness (except for the fixed proportion specification), showing that the practical impact of choosing one utility specification over another, although sizable, is limited. Consequently, main performance deficits are often not due to substitutability or compensatory issues, but rather to real shortfalls in indicator levels, while preference-driven differences tend to be second-order and comparatively robust.

Component (thermodynamics) · Economic indicator · Measure (data warehouse) · Quality (philosophy) · Work (physics) · Capital Investment and Risk Analysis · Health Systems, Economic Evaluations, Quality of Life · Innovation Policy and RD

Tools

Open DOIOpen Access