Skip to main content

ETHNOS_APP

Home • Search • Journals • List 0

Design Experiments

Engaging Policy Makers in the Search for Evidence about What Works

Bibliographic Data

ID6345148
AuthorsGerry Stoker (0000-0002-8172-3395, University of Southampton), Peter John (0000-0002-7934-1187, University of Manchester)
Year2009
Volume57
Issue2
Pages356-373
Publication date2009-06-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenuePolitical Studies (JOURNAL)
Journal identifiersISSN: 0032-3217 • E-ISSN: 1467-9248
PublisherSAGE Publications Inc (PUBLISHER)
DOI10.1111/j.1467-9248.2008.00756.x
OpenAlexW2122528230
LanguageEN
Citations received45
References cited17

This article presents an argument for the greater use of design experiments, which can assist policy making because they provide both robust and timely evidence. We discuss their origins in education research, set out the methodology and propose some adaptations to the techniques used in these education studies to foster their application to a range of policy fields and problem areas. Design experiments need to meet two challenges. Can they provide valid evidence? Can they provide evidence that will be used by policy makers? Our argument shows how design experiments are robust when set against the classical canons of scientific study. We further claim that the design experiment approach offers a more viable means to developing evidence-based policy making than other forms of evaluation because of the timeliness of the insights that it provides

Argument (complex analysis · Data science · Economics · Empirical evidence · Engineering ethics · Epistemology · Evidence-based policy · Management science · Political science · Positive economics · Range (aeronautics · Set (abstract data type · Computer Science · Educational Assessment and Improvement · Engineering · Evaluation and Performance Assessment · School Choice and Performance

  • Industrial Policy

    Lukasz Mamica, Wilfred Dolfsma•Oxford Handbook of State…•2022

  • Governing Climate Change

    Open Access•Andrew J Jordan, Andrew Jordan et al.•Governing Climate Change•2018

  • Democracy and Climate Change

    Open Access•Frederic Hanusch•Democracy and Climate Change•2017

  • Translating Experiments into Policy

    Open Access•Gerry Stoker•The Annals of the American…•2010

  • Citizenship Education for Political Engagement

    Open Access•Steven Donbavand, Bryony Hoskins•Social Sciences•2021

  • Enhancing Collaborative Innovation in the Public Sector

    Open Access•Eva Sørensen, Jacob Torfing•Administration & Society•2011

  • Design science in public administration

    Open Access•Albert Meijer, Albert J Meijer•Perspectives on Public Management…•2024

  • It is about time! Exploring the clashing timeframes of politics and public policy experiments

    Open Access•Ringa Raudla, Külli Sarapuu et al.•Perspectives on Public Management…•2024

  • Theorizing human-centered design doing in public administration

    Open Access•Jodi R Sandfort, Jodi Sandfort•Perspectives on Public Management…•2025

  • Varieties of experimentalism

    Open Access•Christopher Ansell, Christopher K Ansell et al.•Ecological Economics•2016

  • A tool for reflecting on research stances to support sustainability transitions

    Open Access•Laurent Hazard, Marianne Cerf et al.•Nature Sustainability•2019

  • Mutual learning between researchers and farmers during implementation of scientific principles for sustainable development

    Open Access•Laurent Hazard, Patrick Steyaert et al.•Sustainability Science•2017

  • Issue uncertainty and selections of policy instruments in policy pilots

    Open Access•Chengcheng Chen, Jinghua Gao et al.•Journal of Asian Public Policy•2024

  • Are Quantitative Research Methods to Blame for a Growing Irrelevance of Political Science? A Rejoinder to John Trent

    Open Access•Viktoria Kaina, Sebastian Kühn et al.•European Political Science•2014

  • Blockages on the Road to Relevance

    Open Access•Gerry Stoker•European Political Science•2010

  • Innovation or implementation? Local response to low‐carbon policy experimentation in China

    Open Access•Shihong Guo, Qijiao Song et al.•Review of Policy Research•2021

  • Policy experiments

    Open Access•Stefanie Ettelt, Nicholas Mays et al.•Evaluation•2015

  • Tensions of evaluating innovation in a living lab

    Open Access•Romy Dekker, Karin Geuijen et al.•Evaluation•2021

  • What historiography can tell us about evidence-based medicine and policy

    Open Access•Axel Kaehne•Evaluation•2018

  • Experiment-based policy change over time

    Open Access•Roos M Den Uyl, Stefania Munaretto•Global Environmental Change•2020

  • Why are policy innovations rare and so often negative? Blame avoidance and problem denial in climate change policy-making

    Open Access•Michael Howlett•Global Environmental Change•2014

  • From evidence to value-based transition

    Open Access•Laurent Hazard, Nathalie Couix et al.•Agriculture and Human Values•2021

  • Enhancing city vitality by building platforms for interactive political leadership

    Open Access•Eva Sørensen, Jacob Torfing•Cities•2023

  • The impacts of open data initiatives on smart cities

    Open Access•Fátima Trindade Neves, Miguel De Castro Neto et al.•Cities•2020

  • Pragmatic complexity’ a new foundation for moving beyond ‘evidence-based policy making’

    Christopher Ansell, Robert Geyer•Policy Studies•2016

  • Experimentando con un diseño experimental

    Open Access•Judith Bakker, Bas Denters et al.•Revista Internacional de Sociología•2012

  • From ‘trial and error’ to major reform

    Open Access•Andrew S Kelly, Philip Rocco•Public Administration•2019

  • The Benefits and Risks of Experimental Co‐production

    Open Access•Martin Bartenberger, Dawid Sześciło•Public Administration•2015

  • The politics of experimental policymaking

    Open Access•Ringa Raudla, Külli Sarapuu et al.•Policy Sciences•2025

  • The rise of public sector innovation labs

    Open Access•Michael Mcgann, Emma Blomkamp et al.•Policy Sciences•2018

  • The new randomised controlled trials (RCT) movement in public policy

    Open Access•Warren Pearce, Sujatha Raman•Policy Sciences•2014

  • Towards a minimal definition of policy innovation labs

    Open Access•Moshe Maor•Policy Sciences•2026

  • Engaging Experts

    Open Access•Anders Broström, Margaret Mckelvey•Minerva•2017

  • Meaning and power in the design and development of policy experiments

    Open Access•Sreeja Nair, Michael Howlett•Futures•2016

  • Expanding the Use of Experiments on Civic Behavior

    Open Access•Simon Cotterill, Sarah Cotterill et al.•The Annals of the American…•2010

  • Improving democratic governance through institutional design

    Open Access•Chris Skelcher, Jacob Torfing•Regulation & Governance•2010

  • Partecipazione e collaborazione negli interventi di comunità. L'esperienza dei laboratori di quartiere del Comune di Bologna

    Roberta Paltrinieri, Giulia Allegrini•SOCIOLOGIA URBANA E RURALE•2018

  • Applying design science in public policy and administration research

    A Georges L Romme, Albert Meijer et al.•Policy & Politics•2020

  • Social Lobbying

    Christian R Grose, Pamela Lopez et al.•The Journal of Politics•2022

  • Designing Politics

    Open Access•Gerry Stoker•Political Studies Review•2013

  • Exploring the Promise of Experimentation in Political Science

    Open Access•Gerry Stoker•Political Studies•2010

  • Exploring the Contributions of Cultural Theory for Improving Public Deliberation about Complex Policy Problems

    Open Access•Steven Ney, Marco Verweij•Policy Studies Journal•2014

  • Reaping the fruits of co-creation through design experiments

    Open Access•Jacob Torfing, Eva Sørensen et al.•Public Policy and Administration•2023

  • Cognizance and Consultation of Randomized Controlled Trials among Ministerial Policy Analysts

    Open Access•Pierre‐olivier Bédard, Mathieu Ouimet•Review of Policy Research•2012

  • Tailoring Cognitive Mapping Analysis Methods to Different Management Styles of Collective Action by Handling Actor Reasoning Diversity

    Open Access•Gwen Christiansen, Jean Simonneaux et al.•International Journal of…•2022

  • Design Research

    Allan Collins, Diana Joseph et al.•Journal of the Learning Sciences•2004

  • Design Experiments

    Ann L Brown•Journal of the Learning Sciences•1992

  • Toward a Design Science of Education

    Allan Collins•New Directions in Educational…•1992

  • Design Research

    Daniel C Edelson•Journal of the Learning Sciences•2002

  • Resolving social conflicts and field theory in social science

    Kurt Lewin•Resolving social conflicts•1997

  • Social Science and the Evidence-based Policy Movement

    Open Access•Ken Young, Deborah Ashby et al.•Social Policy and Society•2002

  • From Knowing to Doing

    Open Access•Sandra Nutley, Isabel Walter et al.•Evaluation•2003

  • The Practice and the Theory

    Open Access•Margaret Mackenzie, Avril Blamey•Evaluation•2005

  • Design-Based Research

    Open Access•The Design-Based Research Collective•Educational Researcher•2003

  • Design Experiments and Laboratory Approaches to Learning

    Open Access•Bruce D Mccandli, Bruce D McCandliss et al.•Educational Researcher•2003

  • Integrating Curriculum, Instruction, Assessment, and Evaluation in a Technology-Supported Genetics Learning Environment

    Open Access•Daniel T Hickey, Ann C H Kindfield et al.•American Educational Research…•2003

  • Design Experiments in Educational Research

    Open Access•Paul Cobb, Jere Confrey et al.•Educational Researcher•2003

  • Evaluation, Policy Learning and Evidence-Based Policy Making

    Open Access•Ian Sanderson•Public Administration•2002

Unique citing works45
Citations per year2,81
Citation span2010 - 2026 (17)
Citation velocitycurrent
Highly citedNo
Citation typesNeutral: 45

Tools

Open DOISci-Hub
Ethnos_APP • Open Source Project • MIT License • Frontend v2.0.0 • Privacy and Cookies • API Documentation: api.ethnos.app/docs • API Source Code: GitHub • DOI: 10.5281/zenodo.17049435 • Frontend Source Code: GitHub • DOI: 10.5281/zenodo.17050053 • cruz.rio.br • Expectantes Misericordiae