Applying the Renger System Test to Determine Whether a System Evaluation Is Fit for Evaluating a Complex Intervention
A Case Example
Bibliographic Data
| ID | 6414808 |
|---|---|
| Authors | Ralph Renger (0000-0002-0224-1444, Just Evaluation Services, Vail, AZ, USA), Elias Samuels (0000-0002-6725-3382, University of Michigan), Jessica Renger (0000-0002-5120-9504, Kaiser Permanente), Ellen Champagne (University of Michigan) |
| Year | 2025 |
| Volume | 46 |
| Issue | 3 |
| Pages | 353-364 |
| Publication date | 2025-05-14 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | American Journal of Evaluation (JOURNAL) |
| Journal identifiers | ISSN: 1098-2140 • E-ISSN: 1557-0878 |
| Publisher | SAGE Publishing (PUBLISHER • US) |
| DOI | 10.1177/10982140251340680 |
| OpenAlex | W4410352402 |
| Language | EN |
| References cited | 24 |
This article presents the Renger System Test (RST) as a method for assessing whether a system evaluation approach is suitable for evaluating complex interventions. The RST has three criteria: (1) the intervention includes multiple components, (2) these components operate interdependently, and (3) their interdependence produces an outcome that no single component can achieve alone. An NIH-funded case illustrates how to apply the RST to program source documentation. Findings suggest several key terms can be used to infer whether each of the RST criteria is met. The article discusses the pitfalls of using an intervention's label to select the most appropriate evaluation approach, as well as the challenges when there is a disconnect between the funder-mandated evaluation approach and a better fit-for-purpose system approach
Component (thermodynamics · Computer security · Documentation · Intervention (counseling · Key (lock · Management science · Outcome (game theory · Process management · Program evaluation · Psychological intervention · Risk analysis (engineering · Statistics · Test (biology · Computer Science · Engineering · Evaluation and Performance Assessment · Health Policy Implementation Science · Mathematics · Medicine · Primary Care and Health Outcomes · Psychology
Logic models
A Definition of Systems Thinking
How Can Theory-Based Evaluation Make Greater Headway?
Process Flow Mapping for Systems Improvement
Comparing and Contrasting a Program versus System Approach to Evaluation
The Reciprocal Relationship between Implementation Theory and Program Theory in Assisting Program Design and Decision-Making
Constructing and Verifying Program Theory Using Source Documentation
Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions
Systems thinking and evaluation
Advancing Systems Thinking in Evaluation
Beyond Use
Four suppositions about emergent properties of complex interventions that operate and function as systems
System evaluation theory (SET)
Lessons learned in evaluating system interdependencies using qualitative methods
A comparative case study illustrating the influence of a system perspective on the outcome evaluation of simple versus complex interventions
Rapid and Rigorous Qualitative Data Analysis
| Citation velocity | historical |
|---|---|
| Highly cited | No |