Process Tracing and Bayesian Updating for impact evaluation
Bibliographic Data
| ID | 12567464 |
|---|---|
| Authors | Barbara Befani (0000-0002-8379-8905, University of Surrey and University of East Anglia, UK, corresponding author), Gavin Stedman-Bryce (Pamoja Evaluation Services Ltd, UK) |
| Year | 2016 |
| Volume | 23 |
| Issue | 1 |
| Pages | 42-60 |
| Publication date | 2016-06-26 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Evaluation (JOURNAL) |
| Journal identifiers | ISSN: 1356-3890 • E-ISSN: 1461-7153 |
| Publisher | SAGE Publishing (PUBLISHER • US) |
| DOI | 10.1177/1356389016654584 |
| OpenAlex | W2467695307 |
| Language | EN |
| Citations received | 25 |
| References cited | 24 |
Commissioners of impact evaluation often place great emphasis on assessing the contribution made by a particular intervention in achieving one or more outcomes, commonly referred to as a ‘contribution claim’. Current theory-based approaches fail to provide evaluators with guidance on how to collect data and assess how strongly or weakly such data support contribution claims. This article presents a rigorous quali-quantitative approach to establish the validity of contribution claims in impact evaluation, with explicit criteria to guide evaluators in data collection and in measuring confidence in their findings. Coined ‘Contribution Tracing’, the approach is inspired by the principles of Process Tracing and Bayesian Updating, and attempts to make these accessible, relevant and applicable by evaluators. The Contribution Tracing approach, aided by a symbolic ‘contribution trial’, adds value to impact evaluation theory-based approaches by: reducing confirmation bias; improving the conceptual clarity and precision of theories of change; providing more transparency and predictability to data-collection efforts; and ultimately increasing the internal validity and credibility of evaluation findings, namely of qualitative statements. The approach is demonstrated in the impact evaluation of the Universal Health Care campaign, an advocacy campaign aimed at influencing health policy in Ghana
CLARITY · Computer security · Credibility · Data collection · Data science · External validity · Impact evaluation · Management science · Political science · Politics · Process (computing · Process tracing · Sociology · Tracing · Transparency (behavior · Computer Science · Engineering · Evaluation and Performance Assessment · Health Policy Implementation Science · Health Systems, Economic Evaluations, Quality of Life · Medicine · Psychology · Social Psychology
A workflow for collecting and understanding stories at scale, supported by artificial intelligence
Assessing Contributions Collaboratively
Causal mapping for evaluators
Contribution tracing for organisational-wide planning, improved collaboration and evaluative thinking
Testing mechanisms in large-N realistic evaluations
The elusive quest for additionality
Taking Stock of Contribution Analysis
Enduring Themes in John Mayne’s Work
She has become Brave
Weight of Evidence
Policy Effectiveness through Configurational and Mechanistic Lenses
Using process tracing to evaluate unique events
Conversations in Evaluation – Exploring 30 years of debates in our field
Combining process tracing and synthetic control method
A graphical method for causal program attribution in theory-based evaluation
The unused potential of process tracing as evaluation approach
Evaluating business support measures
Diagnostic evaluation with simulated probabilities
The contribution of policy design to realist evaluation
Combining Theories of Change and Realist Evaluation in practice
Diagnostic evaluation and Bayesian Updating
Making rigorous causal claims in a real-life context
Using contribution analysis to evaluate large-scale, transformation change processes
Monitoring and evaluation for thinking and working politically
Impact evaluation with process tracing
Process-Tracing Methods
A Summative Evaluation of RCT Methodology
The Oxford Handbook of Political Methodology
Process Tracing
Addressing Attribution through Contribution Analysis
The contribution of process tracing to theory-based evaluations of complex aid instruments
Contribution analysis
Realistic Evaluation and QCA
A triangulation approach to impact evaluation
Evaluating complex social interventions in a complex world
Making contribution analysis work
The mixing of methods
The Qualitative Method of Impact Analysis
Revising Swiss Policies
Systems Dynamics Modelling in Industrial Development Evaluation
Prosaic or Profound? The Adoption of Systems Ideas by Impact Evaluation
Aiming for Utility in ‘Systems-based Evaluation’
Process Tracing and Contribution Analysis
Understanding Process Tracing
Mixing Methods
| Citations per year | 2.78 |
|---|---|
| Citation span | 2017 - 2025 (9) |
| Highly cited | No |