Benchmarking factor selection and sensitivity

There is an increasing requirement in higher education (HE) worldwide to deliver excellence. Benchmarking is widely used for this purpose, but methodological approaches to the creation of benchmark metrics vary greatly. Approaches require selection of factors for inclusion and subsequent calculation of benchmarks for comparison. We describe an approach using machine learning to select input factors based on their value to predict completion rates of nursing courses. Data from over 36,000 students, from nine institutions over three years were included and weighted averages provided a dynamic baseline for year on year and within year comparisons between institutions. Anonymised outcomes highlight the variation in benchmarked performances between institutions and we demonstrate the value of accompanying sensitivity analyses. Our methods are appropriate worldwide, for many forms of data and at multiple scales of enquiry. We discuss our results in the context of HE management, highlighting the value of scrutinising benchmark calculations

Benchmarking · Business · Economics · Excellence · Higher education · Machine learning · Political science · Computer Science · Engineering · Evaluation and Performance Assessment · Human Resource Development and Performance Evaluation · Psychology · Marketing

• How artificial intelligence (AI) supports nursing education

  Gwo-Jen Hwang, Kai‐Yu Tang et al.•Interactive Learning Environments•2024

  Incluir item

• Development of an early warning system for higher education institutions by predicting first‐year student academic performance

  Open Access•Cem Recai Çırak, Hakan Akıllı et al.•Higher Education Quarterly•2024

  Incluir item

• Evaluating Higher Education Performance via Machine Learning During Disruptive Times

  Open Access•Semih Sait Yılmaz, Ayse Collins et al.•European Journal of Education•2024

  Incluir item

• A case for using both direct and indirect benchmarking to compare university performance metrics

  W E Harris, A Mark Langan et al.•Studies in Higher Education•2019

  Incluir item

• Rankings and the Reshaping of Higher Education

  Open Access•Ellen Hazelkorn•Rankings and the Reshaping of…•2015

  Incluir item

• Variable Importance Assessment in Regression

  Ulrike Grömping•The American Statistician•2009

  Incluir item

• An introduction to recursive partitioning

  Carolin Strobl, James D Malley et al.•Psychological Methods•2009

  Incluir item

• Rickety numbers

  Open Access•Michaela Saisana, Beatrice D’hombres et al.•Research Policy•2011

  Incluir item

• Variable selection using random forests

  Open Access•Robin Genuer, Jean‐michel Poggi et al.•Pattern Recognition Letters•2010

  Incluir item

• Random Forests

  Open Access•Leo Breiman•Machine Learning•2001

  Incluir item

• Benchmarking universities' efficiency indicators in the presence of internal heterogeneity

  Tommaso Agasisti, Francesca Bonomi•Studies in Higher Education•2014

  Incluir item

• Pitfalls in aggregating performance measures in higher education

  Ross Williams, Gaétan De Rassenfosse•Studies in Higher Education•2016

  Incluir item