John Mayne
Biographic Data
| ID | 4000487 |
|---|---|
| NAME | John Mayne |
| GIVEN NAMES | John |
| FAMILY NAME | Mayne |
| SIGNATURE | MAYNE J |
| AFFILIATIONS | Ottawa, Ontario |
| VERIFIED | No |
| TOTAL WORKS | 30 |
| TOTAL CITATIONS | 157 |
| AUTHOR COUNT | 30 |
| EDITOR COUNT | 0 |
| FIRST PUBLICATION YEAR | 1986 |
| LATEST PUBLICATION YEAR | 2023 |
| H-INDEX | 6 |
Assumptions in theories of change
Building evaluative culture in community services
Sustainability Analysis of Intervention Benefits
Theories of change (ToCs) are models of how an intervention is expected to bring about changes in results of interest—the benefits. Robust ToCs spell out the conditions under which the intervention should “work,” that is, bring about or contribute to the desired benefits. But rarely does the ToC address issues of the future sustainability of the benefits from the intervention. The article discusses how ToCs can adopt a sustainability lens and, al…
Using Actor-Based Theories of Change to Conduct Robust Evaluation in Complex Settings
The use of theories of change (ToCs) is a hallmark of sound evaluation practice. As interventions have become more complex, the development of ToCs that adequately unpack this complexity has become more challenging. Equally important is the development of evaluable ToCs, necessary for conducting robust theory-based evaluation approaches such as contribution analysis (CA). This article explores one approach to tackling these challenges through the…
Revisiting Contribution Analysis
The basic ideas behind contribution analysis were set out in 2001. Since then, interest in the approach has grown and contribution analysis has been operationalized in different ways. In addition, several reviews of the approach have been published and raise a few concerns. In this article, I clarify several of the key concepts behind contribution analysis, including contributory causes and contribution claims. I discuss the need for reasonably r…
Linking Evaluation to Expenditure Reviews
It is frequently assumed and not contested that evaluation should play a significant role in budgeting and, more specifically, in expenditure reviews. This article argues otherwise: that evaluation is neither fit nor designed to play such a role. Rather, if there is a desire by budget officials for credible evidence on the performance of interventions, then they need to invest in a different form of evaluation, namely, expenditure evaluations, se…
Théories du changement
Although theories of change are frequently discussed in the evaluation literature and there is general agreement on what a theory of change is conceptually, there is actually little agreement beyond the big picture of just what a theory of change comprises, what it shows, how it can be represented, and how it can be used. This article outlines models for theories of change and their development that have proven quite useful for both straightforwa…
Theory of Change Analysis
Models for theories of change vary widely as do how they are used. What constitutes a good or robust theory of change has not been discussed much. This article sets out and discusses criteria for robust theories of change. As well, it discusses how these criteria can be used to undertake a vigorous assessment of a theory of change. A solid analysis of a theory of change can be extremely useful, both for designing or assessing the designs of an in…
Evaluating complex interventions
There is a growing recognition that programs that seek to change people’s lives are intervening in complex systems, which puts a particular set of requirements on program monitoring and evaluation. Developing complexity-aware program monitoring and evaluation systems within existing organizations is difficult because they challenge traditional orthodoxy. Little has been written about the practical experience of doing so. This article describes th…
Useful Theory of Change Models
Although theories of change are frequently discussed in the evaluation literature and there is general agreement on what a theory of change is conceptually, there is actually little agreement beyond the big picture of just what a theory of change comprises, what it shows, how it can be represented, and how it can be used. This article outlines models for theories of change and their development that have proven quite useful for both straightforwa…
Using theories of change in the CGIAR Research Program on Agriculture for Nutrition and Health
Theories of change are increasingly being discussed and referenced in development evaluation even while the elements of what a theory of change consist of differ widely among applications. Equally, examples of actual use of theories of change other than as overview illustrations of interventions are rare. In this article, we present generic models of theories of change for both straightforward and more complex interventions. A number of examples …
Process Tracing and Contribution Analysis
This article proposes a combination of a popular evaluation approach, contribution analysis (CA), with an emerging method for causal inference, process tracing (PT). Both are grounded in generative causality and take a probabilistic approach to the interpretation of evidence. The combined approach is tested on the evaluation of the contribution of a teaching programme to the improvement of school performance of girls, and is shown to be preferabl…
Contribution analysis
In this introductory article, a brief history and introduction to contribution analysis is provided to lay the stage for the articles that follow. At the heart of contribution analysis is the aim to be able to make credible causal claims about the contribution an intervention is making to observed results. The key role that theories of change play is noted, and what a useful theory of change ought to contain is discussed. The article then makes a…
Broadening the range of designs and methods for impact evaluations
Introducing the study'rigorous' experimental methodology) and a 'substantive definition' (the DAC's, that does not require a particular methodology
A Commentary on Shepherd’s “In Search of a Balanced Canadian Federal Evaluation Function
Building an Evaluative Culture
As many reviews of results-based performance systems have noted, a weak evaluative culture in an organization undermines attempts at building an effective evaluation and/or results management regime. This article sets out what constitutes a strong evaluative culture where information on performance results is deliberately sought in order to learn how to better manage and deliver programs and services. Such an organization values empirical evidenc…
The Lay of the Land
A group of 12 evaluation practitioners and observers takes stock of the state of program evaluation in Canada. Each contributor provides a personal viewpoint, based on his or her own experience in the field. The selection of contributors constitutes a purposive sample aimed at providing depth of view and a variety of perspectives. Each presentation highlights one strength of program evaluation practiced in Canada, one weakness, one threat, and on…
Guest Editor’s Introduction
Studies Are Not Enough
The authors contend that developments in the public and not-for-profit sectors over the last decade or so have profound implications for the profession of evaluation, implications that are not being adequately addressed. They argue that evaluation needs to transform itself if it wishes to play a significant role in the management of organizations in these sectors. Beyond traditional evaluation studies, evaluators working in public and not-for-pro…
Challenges and Lessons in Implementing Results-Based Management
Integrating performance information into budgeting, managing and reporting has become a common component of good public and not-for-profit management. In many jurisdictions, efforts to do so have been under way for many years, yet progress is usually seen as slow at best. It is also clear that, while much has been learned, many challenges remain; few organizations would argue they have been completely successful. The paper argues that implementin…
Performance Studies
Typically, a good measurement strategy to support results-based management includes both ongoing performance measures and periodic evaluations. It is argued that this is a too limited set of measurement tools, resulting, not infrequently, in less useful and costly ongoing performance measures. It is proposed that, in addition to ongoing performance measures and periodic evaluations, an alternative measurement tool called a performance study shoul…
Audit and Evaluation in Public Management
Audit and evaluation play important roles in public management. There can be confusion and debate, however, over what each of these functions cover and indeed what roles they should play. This article reviews and compares each in relation to public management and the key challenges each face in today’s public sector. Audit and evaluation should play different roles in public management and provide different information on the performance of publi…
The Lay of the Land
A group of 12 evaluation practitioners and observers takes stock of the state of program evaluation in Canada. Each of the contributors provides a personal viewpoint, based on their own experience in the field. The selection of contributors constitutes a purposive sample aimed at providing depth of view and a variety of perspectives. Each presentation highlights one strength of program evaluation practiced in Canada, one weakness, one threat, and…
Reporting on Outcomes
Results, and more particularly outcomes, are at the centre of public management reform in many jurisdictions, including Canada. Managing for outcomes, including setting realistic outcome expectations for programs, and credibly reporting on what was achieved are proving to be challenges, perhaps not unexpectedly, given the challenges faced in evaluating the outcomes of public programs. This article discusses how the use of results chains can assis…
Addressing Attribution through Contribution Analysis
The changing culture of public administration involves accountability for results and outcomes. This article suggests that performance measurement can address such attribution questions. Contribution analysis has a major role to play in helping managers, researchers, and policymakers to arrive at conclusions about the contribution their program has made to particular outcomes. The article describes the steps necessary to produce a credible contri…
Broadening the range of designs and methods for impact evaluations
Introducing the study'rigorous' experimental methodology) and a 'substantive definition' (the DAC's, that does not require a particular methodology
Contribution analysis
In this introductory article, a brief history and introduction to contribution analysis is provided to lay the stage for the articles that follow. At the heart of contribution analysis is the aim to be able to make credible causal claims about the contribution an intervention is making to observed results. The key role that theories of change play is noted, and what a useful theory of change ought to contain is discussed. The article then makes a…
Challenges and Lessons in Implementing Results-Based Management
Integrating performance information into budgeting, managing and reporting has become a common component of good public and not-for-profit management. In many jurisdictions, efforts to do so have been under way for many years, yet progress is usually seen as slow at best. It is also clear that, while much has been learned, many challenges remain; few organizations would argue they have been completely successful. The paper argues that implementin…
Using theories of change in the CGIAR Research Program on Agriculture for Nutrition and Health
Theories of change are increasingly being discussed and referenced in development evaluation even while the elements of what a theory of change consist of differ widely among applications. Equally, examples of actual use of theories of change other than as overview illustrations of interventions are rare. In this article, we present generic models of theories of change for both straightforward and more complex interventions. A number of examples …
Process Tracing and Contribution Analysis
This article proposes a combination of a popular evaluation approach, contribution analysis (CA), with an emerging method for causal inference, process tracing (PT). Both are grounded in generative causality and take a probabilistic approach to the interpretation of evidence. The combined approach is tested on the evaluation of the contribution of a teaching programme to the improvement of school performance of girls, and is shown to be preferabl…
Evaluating complex interventions
There is a growing recognition that programs that seek to change people’s lives are intervening in complex systems, which puts a particular set of requirements on program monitoring and evaluation. Developing complexity-aware program monitoring and evaluation systems within existing organizations is difficult because they challenge traditional orthodoxy. Little has been written about the practical experience of doing so. This article describes th…
Public Power Outside Government
In Defense of Program Evaluation
Défense de l'évaluation de programme
Le 12 mars 1986 et les jours suivants, les journaux canadiens ont publié de nombreux articles sur le Rapport final (1986) du Groupe de travail ministériel chargé de l'examen des programmes. Plusieurs journalistes ont tenu des propos peu flatteurs sur l'évaluation des programmes fédéraux et ont mentionné la «médiocrité», l'«inutilité» et l'«indulgence» des évaluations. Les articles de Christopher Waddell et Hugh Windsor, du 12 mars, et de Bruce Li…
Ongoing Program Performance Information Systems and Program Evaluation in the Government of Canada
Les systèmes d'information sur la performance des programmes n'ont pas donné, dans bien des cas, les résultats attendus, et cela parce qu'on n'a pas mis le soin voulu pour déterminer quels étaient les renseignements nécessaires et qui avait besoin de ces renseignements, et pour établir le lien entre ces éléments et les diverses caractéristiques des renseignements périodiques par rapport aux renseignements permanents. L'information permanente est …
Public Power Outside Government
Accountability for Collaborative Programme Delivery Arrangements in Canada’s Federal Government
Addressing Attribution through Contribution Analysis
The changing culture of public administration involves accountability for results and outcomes. This article suggests that performance measurement can address such attribution questions. Contribution analysis has a major role to play in helping managers, researchers, and policymakers to arrive at conclusions about the contribution their program has made to particular outcomes. The article describes the steps necessary to produce a credible contri…
The Lay of the Land
A group of 12 evaluation practitioners and observers takes stock of the state of program evaluation in Canada. Each of the contributors provides a personal viewpoint, based on their own experience in the field. The selection of contributors constitutes a purposive sample aimed at providing depth of view and a variety of perspectives. Each presentation highlights one strength of program evaluation practiced in Canada, one weakness, one threat, and…
Reporting on Outcomes
Results, and more particularly outcomes, are at the centre of public management reform in many jurisdictions, including Canada. Managing for outcomes, including setting realistic outcome expectations for programs, and credibly reporting on what was achieved are proving to be challenges, perhaps not unexpectedly, given the challenges faced in evaluating the outcomes of public programs. This article discusses how the use of results chains can assis…
Performance Studies
Typically, a good measurement strategy to support results-based management includes both ongoing performance measures and periodic evaluations. It is argued that this is a too limited set of measurement tools, resulting, not infrequently, in less useful and costly ongoing performance measures. It is proposed that, in addition to ongoing performance measures and periodic evaluations, an alternative measurement tool called a performance study shoul…
Audit and Evaluation in Public Management
Audit and evaluation play important roles in public management. There can be confusion and debate, however, over what each of these functions cover and indeed what roles they should play. This article reviews and compares each in relation to public management and the key challenges each face in today’s public sector. Audit and evaluation should play different roles in public management and provide different information on the performance of publi…
Guest Editor’s Introduction
Studies Are Not Enough
The authors contend that developments in the public and not-for-profit sectors over the last decade or so have profound implications for the profession of evaluation, implications that are not being adequately addressed. They argue that evaluation needs to transform itself if it wishes to play a significant role in the management of organizations in these sectors. Beyond traditional evaluation studies, evaluators working in public and not-for-pro…
Challenges and Lessons in Implementing Results-Based Management
Integrating performance information into budgeting, managing and reporting has become a common component of good public and not-for-profit management. In many jurisdictions, efforts to do so have been under way for many years, yet progress is usually seen as slow at best. It is also clear that, while much has been learned, many challenges remain; few organizations would argue they have been completely successful. The paper argues that implementin…
Building an Evaluative Culture
As many reviews of results-based performance systems have noted, a weak evaluative culture in an organization undermines attempts at building an effective evaluation and/or results management regime. This article sets out what constitutes a strong evaluative culture where information on performance results is deliberately sought in order to learn how to better manage and deliver programs and services. Such an organization values empirical evidenc…
The Lay of the Land
A group of 12 evaluation practitioners and observers takes stock of the state of program evaluation in Canada. Each contributor provides a personal viewpoint, based on his or her own experience in the field. The selection of contributors constitutes a purposive sample aimed at providing depth of view and a variety of perspectives. Each presentation highlights one strength of program evaluation practiced in Canada, one weakness, one threat, and on…
A Commentary on Shepherd’s “In Search of a Balanced Canadian Federal Evaluation Function
Contribution analysis
In this introductory article, a brief history and introduction to contribution analysis is provided to lay the stage for the articles that follow. At the heart of contribution analysis is the aim to be able to make credible causal claims about the contribution an intervention is making to observed results. The key role that theories of change play is noted, and what a useful theory of change ought to contain is discussed. The article then makes a…
Broadening the range of designs and methods for impact evaluations
Introducing the study'rigorous' experimental methodology) and a 'substantive definition' (the DAC's, that does not require a particular methodology
Process Tracing and Contribution Analysis
This article proposes a combination of a popular evaluation approach, contribution analysis (CA), with an emerging method for causal inference, process tracing (PT). Both are grounded in generative causality and take a probabilistic approach to the interpretation of evidence. The combined approach is tested on the evaluation of the contribution of a teaching programme to the improvement of school performance of girls, and is shown to be preferabl…
Useful Theory of Change Models
Although theories of change are frequently discussed in the evaluation literature and there is general agreement on what a theory of change is conceptually, there is actually little agreement beyond the big picture of just what a theory of change comprises, what it shows, how it can be represented, and how it can be used. This article outlines models for theories of change and their development that have proven quite useful for both straightforwa…
Using theories of change in the CGIAR Research Program on Agriculture for Nutrition and Health
Theories of change are increasingly being discussed and referenced in development evaluation even while the elements of what a theory of change consist of differ widely among applications. Equally, examples of actual use of theories of change other than as overview illustrations of interventions are rare. In this article, we present generic models of theories of change for both straightforward and more complex interventions. A number of examples …
Théories du changement
Although theories of change are frequently discussed in the evaluation literature and there is general agreement on what a theory of change is conceptually, there is actually little agreement beyond the big picture of just what a theory of change comprises, what it shows, how it can be represented, and how it can be used. This article outlines models for theories of change and their development that have proven quite useful for both straightforwa…
Theory of Change Analysis
Models for theories of change vary widely as do how they are used. What constitutes a good or robust theory of change has not been discussed much. This article sets out and discusses criteria for robust theories of change. As well, it discusses how these criteria can be used to undertake a vigorous assessment of a theory of change. A solid analysis of a theory of change can be extremely useful, both for designing or assessing the designs of an in…
Evaluating complex interventions
There is a growing recognition that programs that seek to change people’s lives are intervening in complex systems, which puts a particular set of requirements on program monitoring and evaluation. Developing complexity-aware program monitoring and evaluation systems within existing organizations is difficult because they challenge traditional orthodoxy. Little has been written about the practical experience of doing so. This article describes th…
Linking Evaluation to Expenditure Reviews
It is frequently assumed and not contested that evaluation should play a significant role in budgeting and, more specifically, in expenditure reviews. This article argues otherwise: that evaluation is neither fit nor designed to play such a role. Rather, if there is a desire by budget officials for credible evidence on the performance of interventions, then they need to invest in a different form of evaluation, namely, expenditure evaluations, se…
Evaluation and Performance Assessment (23 works) · Psychology (19 works) · Computer Science (18 works) · Political science (18 works) · Business (15 works) · Economics (12 works) · Sociology (12 works) · Public relations (10 works) · Engineering (8 works) · Public Policy and Administration Research (8 works)