Governments harnessing the power of data to get 'value for money
A simulation study of England's Office for Students B3 Proceed Metric
Bibliographic Data
| ID | 4098812 |
|---|---|
| Authors | Alexander Bradley (0000-0003-4304-7653, University of Portsmouth, corresponding author), Mark Quigley (0000-0003-4342-1369, Swansea University) |
| Year | 2023 |
| Volume | 48 |
| Issue | 8 |
| Pages | 1289-1302 |
| Publication date | 2023-08-03 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Studies in Higher Education (JOURNAL) |
| Journal identifiers | ISSN: 0307-5079 • E-ISSN: 1470-174X |
| Publisher | Informa UK Limited (PUBLISHER • GB) |
| DOI | 10.1080/03075079.2023.2196292 |
| OpenAlex | W4362554957 |
| Language | EN |
| Citations received | 6 |
| References cited | 18 |
The mass participation in higher education has led to greater spending by governments and students which has increased the focus on graduate outcomes. In England, the Office for Students (OfS) is planning to take regulatory action, using the Proceed metric, against universities and their courses which do not have 60% of students with positive outcomes within 15 months of graduation. This study uses simulations to explore how effectively the Proceed metric can (a) identify the true population level of positive outcomes, (b) explore the precision of those estimates, and (c) accurately classify courses below the 60% threshold. The simulation varied: level of positive outcomes within the population (20–90%), sample size (40–1000), and the percentage of the population sampled (30–90%). The bias (difference between sample and population estimate), coverage probability (proportion of true population estimates within the intervals), and precision of confidence intervals (average range of confidence intervals) were calculated. Three main findings were (a) the Proceed metric is accurate in terms of bias and coverage probability, (b) the estimates lack precision, especially with small sample sizes, and (c) the imprecision will impact the Proceed metric ability to correctly classify courses below the 60% threshold. Governments seeking to collect graduate survey data and use it to regulate universities should make every effort to maximise sample size and should resist the temptation to regulate at a micro-level (i.e. courses) since classification will likely be inaccurate, especially for those just below the threshold
Actuarial science · Business · Confidence interval · Econometrics · Economics · Population · Sample size determination · Sociology · Statistics · Computer Science · Demography · Healthcare cost, quality, practices · Healthcare Operations and Scheduling Optimization · Healthcare Policy and Management · Mathematics · Psychology · Marketing
Intolerance of uncertainty and its impact on university students’ confidence, awareness and experience with employer selection tests
Quality Assurance in Private Higher Education
Rethinking curriculum for uncertain futures
The definition of educational quality in a quasi-marketised higher education system
Transformational accounts of students’ undergraduate education are evoked by their engagement with knowledge
Is England’s Office for Students likely to falsely identify courses as below threshold on the B3 progression metric
The incidence and effects of overeducation in the U.K. graduate labour market
Bologna Process and Its Impact on Spanish Graduates Employability
A Policy Reform Aimed at Improving the Job Quality of Graduates
The degree is not enough’
Estimating the Effect of Incentives on Mail Survey Response Rates
What we know and what we need to know about graduate skills
| Unique citing works | 6 |
|---|---|
| Citations per year | 6 |
| Citation span | 2025 - 2026 (2) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 6 |