How Credible Is the Evidence, and Does It Matter? An Analysis of the Program Assessment Rating Tool

Dados Bibliográficos

ID6366459
AutoresCarolyn J Heinrich (0000-0002-7731-9138, The University of Texas at Austin, autor correspondente)
Ano2011
Volume72
Fascículo1
Páginas123-134
Data de publicação2011-11-11
Peer ReviewedSim
Open AccessSim
TipoARTICLE
PeriódicoPublic Administration Review (JOURNAL)
Identificadores do periódicoISSN: 0033-3352 • E-ISSN: 1540-6210
EditoraWiley (PUBLISHER • GB)
DOI10.1111/j.1540-6210.2011.02490.x
OpenAlexW2030699263
IdiomaEN
Citações recebidas39
Referências citadas11

This research empirically assesses the quality of evidence that agencies provided to the Office of Management and Budget in the application of the Program Assessment Rating Tool (PART), introduced in 2002 to more rigorously, systematically, and transparently assess public program effectiveness and hold agencies accountable for results by tying them to the executive budget formulation process and program funding. Evidence submitted by 95 programs administered by the U.S. Department of Health and Human Services for the PART assessment is analyzed using measures that capture the quality of evidence and methods used by programs and information on characteristics of agencies that might relate to program results and government funding decisions. The study finds that of those programs offering some evidence, most was internal and qualitative, and about half did not assess how their performance compared to other government or private programs with similar objectives. Programs were least likely to provide externally generated evidence of their performance relative to long‐term and annual performance goals. Importantly, overall PART and results scores were (statistically) significantly lower for programs that failed to provide quantitative evidence and did not use long‐term measures, baseline measures or targets, or independent evaluations. Although the PART program results ratings and overall PART scores had no discernible consequences for program funding over time, the PART assessments appeared to take seriously the evaluation of evidence quality, a positive step forward in recent efforts to base policy decisions on more rigorous evidence

Actuarial science · Baseline (sea · Business · Economics · Government (linguistics · Political science · Program evaluation · Quality (philosophy · Tying · Evaluation and Performance Assessment · Healthcare Policy and Management · Public Administration · Public Policy and Administration Research