Thomas D Cook
Biographic Data
| ID | 266196 |
|---|---|
| NAME | Thomas D Cook |
| GIVEN NAMES | Thomas D |
| FAMILY NAME | Cook |
| SIGNATURE | COOK T D |
| AFFILIATIONS | Northwestern University |
| ORCID | 0000-0002-3428-4852 |
| VERIFIED | Yes |
| TOTAL WORKS | 77 |
| TOTAL CITATIONS | 645 |
| AUTHOR COUNT | 77 |
| EDITOR COUNT | 0 |
| FIRST PUBLICATION YEAR | 1968 |
| LATEST PUBLICATION YEAR | 2024 |
| H-INDEX | 13 |
How Consistent Are Meanings of “Evidence-Based”? A Comparative Review of 12 Clearinghouses that Rate the Effectiveness of Educational Programs
Clearinghouses set standards of scientific quality to vet existing research to determine how “evidence-based” an intervention is. This paper examines 12 educational clearinghouses to describe their effectiveness criteria, to estimate how consistently they rate the same program, and to probe why their judgments differ. All the clearinghouses value random assignment, but they differ in how they treat its implementation, how they weight quasi-experi…
Combining a Local Comparison Group, a Pretest Measure, and Rich Covariates: How Well Do They Collectively Reduce Bias in Nonequivalent Comparison Group Designs
This study examined bias reduction in the eight nonequivalent comparison group designs (NECGDs) that result from combining (a) choice of a local versus non-local comparison group, and analytic use or not of (b) a pretest measure of the study outcome and (c) a rich set of other covariates. Bias was estimated as the difference in causal estimate between each NECGD and a carefully appraised randomized experiment with the same intervention, outcome, …
Internal and External Validity of the Comparative Interrupted Time‐series Design: A Meta‐analysis
This paper meta‐analyzes 12 heterogeneous studies that examine bias in the comparative interrupted time‐series design (CITS) that is often used to evaluate the effects of social policy interventions. To measure bias, each CITS impact estimate was differenced from the estimate derived from a theoretically unbiased causal benchmark study that tested the same hypothesis with the same treatment group, outcome data, and estimand. In 10 studies, the be…
Manifesto for new directions in developmental science
Although developmental science has always been evolving, these times of fast-paced and profound social and scientific changes easily lead to disorienting fragmentation rather than coherent scientific advances. What directions should developmental science pursue to meaningfully address real-world problems that impact human development throughout the lifespan? What conceptual or policy shifts are needed to steer the field in these directions? The p…
Twenty-six assumptions that have to be met if single random assignment experiments are to warrant “gold standard” status: A commentary on Deaton and Cartwright
The Internal and External Validity of the Regression Discontinuity Design: A Meta‐analysis of 15 Within‐study Comparisons
Theory predicts that regression discontinuity (RD) provides valid causal inference at the cutoff score that determines treatment assignment. One purpose of this paper is to test RD's internal validity across 15 studies. Each of them assesses the correspondence between causal estimates from an RD study and a randomized control trial (RCT) when the estimates are made at the same cutoff point where they should not differ asymptotically. However, sta…
Redefine statistical significance
William Raymond Shadish Jr: Will Shadish's Intellectual Accomplishments
Quasi‐Experimental Design
Quasi‐experiments usually test the causal consequences of long‐lasting treatments outside of the laboratory. But unlike “true” experiments where treatment assignment is at random, assignment in quasi‐experiments is by self‐selection or administrator judgment.
Generalization: Conceptions in the Social Sciences
Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation
A decade ago, the Society of Prevention Research (SPR) endorsed a set of standards for evidence related to research on prevention interventions. These standards (Flay et al., Prevention Science 6:151-175, 2005) were intended in part to increase consistency in reviews of prevention research that often generated disparate lists of effective interventions due to the application of different standards for what was considered to be necessary to demons…
Causally Valid Relationships That Invoke the Wrong Causal Agent: Construct Validity of the Cause in Policy Research
Internal validity relates to whether the relationship between two variables is causal. Various identification devices are used to infer the validity of this relationship, such as random assignment or regression discontinuity. However, these devices are irrelevant to validly identifying the name, label, or word that should be attached to the cause or to the effect. Yet, a valid causal relationship has little practical value for theory development …
Examining the Internal Validity and Statistical Precision of the Comparative Interrupted Time Series Design by Comparison With a Randomized Experiment
Although evaluators often use an interrupted time series (ITS) design to test hypotheses about program effects, there are few empirical tests of the design’s validity. We take a randomized experiment on an educational topic and compare its effects to those from a comparative ITS (CITS) design that uses the same treatment group as the experiment but a nonequivalent comparison group that is assessed at six time points before treatment. We estimate …
Generalizing Causal Knowledge in the Policy Sciences: External Validity as a Task of Both Multiattribute Representation and Multiattribute Extrapolation
Big Data” in Research on Social Policy: Point/Counterpoint
Strengthening the Regression Discontinuity Design Using Additional Design Elements: A Within‐study Comparison
The sharp regression discontinuity design (RDD) has three key weaknesses compared to the randomized clinical trial (RCT). It has lower statistical power, it is more dependent on statistical modeling assumptions, and its treatment effect estimates are limited to the narrow subpopulation of cases immediately around the cutoff, which is rarely of direct scientific or policy interest. This paper examines how adding an untreated comparison to the basi…
Research Designs for Program Evaluation
Over the past two decades, the program evaluation literature has made great advances on improving methodological approaches for establishing causal inference. The two most significant developments include establishing the primacy of design over statistical adjustment procedures for making causal inferences, and using potential outcomes to specify the exact causal estimands produced by the research designs. This chapter presents four research desi…
Randomized Experiments in Educational Research
The importance of covariate selection in controlling for selection bias in observational studies.
The assumption of strongly ignorable treatment assignment is required for eliminating selection bias in observational studies. To meet this assumption, researchers often rely on a strategy of selecting covariates that they think will control for selection bias. Theory indicates that the most important covariates are those highly correlated with both the real selection process and the potential outcomes. However, when planning a study, it is rarel…
How Bias Reduction Is Affected by Covariate Choice, Unreliability, and Mode of Data Analysis: Results From Two Types of Within-Study Comparisons
This study uses within-study comparisons to assess the relative importance of covariate choice, unreliability in the measurement of these covariates, and whether regression or various forms of propensity score analysis are used to analyze the outcome data. Two of the within-study comparisons are of the four-arm type, and many more are of the three-arm type. To examine unreliability, simulations of differences in reliability are deliberately intro…
Contemporary Thinking About Causation in Evaluation: A Dialogue With Tom Cook and Michael Scriven
Legitimate knowledge claims about causation have been a central concern among evaluators and applied researchers for several decades and often have been the subject of heated debates. In recent years these debates have resurfaced with a renewed intensity, due in part to the priority currently being given to randomized experiments by many funders of evaluation studies, such as the Institute for Educational Sciences. In this dialogue, which took pl…
Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within‐study comparisons
This paper analyzes 12 recent within‐study comparisons contrasting causal estimates from a randomized experiment with those from an observational study sharing the same treatment group. The aim is to test whether different causal estimates result when a counterfactual group is formed, either with or without random assignment, and when statistical adjustments for selection are made in the group from which random assignment is absent. We identify t…
Some empirically viable alternatives to random assignment
Comer’s School Development Program in Chicago: Effects on Involvement With the Juvenile Justice System From the Late Elementary Through the High School Years
In 2000, Cook, Murphy, and Hunt published a multilevel study of Chicago inner-city schools in order to evaluate James Comer’s School Development Program (SDP). One main finding was that SDP reduced the rate of change and final posttest mean when delinquency was assessed annually between Grades 5 and 8 using a self-report measure of acting out. The present study examined whether these same mean and slope effects would be observed when delinquency …
Friendship Influences During Early Adolescence: The Special Role of Friends' Grade Point Average
This study examines how a wide variety of diverse friendship group attributes affect changes in indicators of school performance, social behavior, and mental health between early seventh and late eighth grade. Nine hundred and one middle school students named their friends. Independent data from these friends were used to construct friendship groups that were then characterized in terms of their mean level on measures of academic performance, soc…
Redefine statistical significance
Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within‐study comparisons
This paper analyzes 12 recent within‐study comparisons contrasting causal estimates from a randomized experiment with those from an observational study sharing the same treatment group. The aim is to test whether different causal estimates result when a counterfactual group is formed, either with or without random assignment, and when statistical adjustments for selection are made in the group from which random assignment is absent. We identify t…
Qualitative and quantitative methods in evaluation research
An effectiveness‐based evaluation of five state pre‐kindergarten programs
Since 1980, the number of state pre‐kindergarten (pre‐K) programs has more than doubled, with 38 states enrolling more than one million children in 2006 alone. This study evaluates how five state pre‐K programs affected children's receptive vocabulary, math, and print awareness skills. Taking advantage of states' strict enrollment policies determined by a child's date of birth, a regression‐discontinuity design was used to estimate effects in Mic…
Explaining Aspects of the Transition to Adulthood in Italy, Sweden, Germany, and the United States: A Cross-Disciplinary, Case Synthesis Approach
This paper synthesizes essays on Italy, Sweden, Germany, and the United States that were presented at a conference seeking to explain the school, work, and family findings outlined in these foregoing chapters. Three essays were written per country--by a social historian, by a developmental scientist, and by someone in social policy. This paper synthesizes these country-specific accounts. For Italy, the synthesis constructed stresses the accommoda…
Subject effects in laboratory research: An examination of subject roles, demand characteristics, and valid inference
Comer's School Development Program in Chicago: A Theory-Based Evaluation
Using fifth through eighth-grade students, the Comer School Development Program was evaluated in 10 inner city Chicago schools over 4 years, contrasting them with nine randomly selected no-treatment comparison schools. Comer schools implemented more program details than the controls but were not faithful to all program particulars. Students' ratings of the school's social climate improved soon after the program began. By the last 2 study years, b…
The Implicit Assumptions of Television Research: An Analysis of the 1982 Nimh Report on Television and Behavior
The authors analyze some of the assumptions underlying most current research on television. They emphasize the dependence on (1) an individual rather than an institutional level of analysis; (2) a model of research utilization that pays little explicit attention to where sources of leverage lie for changes in programming; (3) extremely simple models of the selection processes associated with different levels of television viewing; and (4) uncriti…
Comer's School Development Program in Prince George's County, Maryland: A Theory-Based Evaluation
A randomized experiment of Comer's School Development Program was conducted in 23 middle schools in Prince George's County, Maryland. The school population is predominantly African American, with considerable internal variation in household socioeconomic standing. This study involved repeated measurement with more than 12,000 students and 2,000 staff a survey of more than 1,000 parents, and extensive access to student records. It showed that Come…
History of the sleeper effect: Some logical pitfalls in accepting the null hypothesis
Equity theory and the cognitive ability of children
The Internal and External Validity of the Regression Discontinuity Design: A Meta‐analysis of 15 Within‐study Comparisons
Theory predicts that regression discontinuity (RD) provides valid causal inference at the cutoff score that determines treatment assignment. One purpose of this paper is to test RD's internal validity across 15 studies. Each of them assesses the correspondence between causal estimates from an RD study and a randomized control trial (RCT) when the estimates are made at the same cutoff point where they should not differ asymptotically. However, sta…
Strengthening the Regression Discontinuity Design Using Additional Design Elements: A Within‐study Comparison
The sharp regression discontinuity design (RDD) has three key weaknesses compared to the randomized clinical trial (RCT). It has lower statistical power, it is more dependent on statistical modeling assumptions, and its treatment effect estimates are limited to the narrow subpopulation of cases immediately around the cutoff, which is rarely of direct scientific or policy interest. This paper examines how adding an untreated comparison to the basi…
Examining the Internal Validity and Statistical Precision of the Comparative Interrupted Time Series Design by Comparison With a Randomized Experiment
Although evaluators often use an interrupted time series (ITS) design to test hypotheses about program effects, there are few empirical tests of the design’s validity. We take a randomized experiment on an educational topic and compare its effects to those from a comparative ITS (CITS) design that uses the same treatment group as the experiment but a nonequivalent comparison group that is assessed at six time points before treatment. We estimate …
Friendship Influences During Early Adolescence: The Special Role of Friends' Grade Point Average
This study examines how a wide variety of diverse friendship group attributes affect changes in indicators of school performance, social behavior, and mental health between early seventh and late eighth grade. Nine hundred and one middle school students named their friends. Independent data from these friends were used to construct friendship groups that were then characterized in terms of their mean level on measures of academic performance, soc…
The Comer School Development Program: A Theoretical Analysis
Practice resting on theories of society, of needs, of social relationships: applications
Why have Educational Evaluators Chosen Not to Do Randomized Experiments
This article analyzes the reasons that have been adduced within the community of educational evaluators for not doing randomized experiments. The objections vary in cogency. Those that have most substance are not insurmountable, however, and strategies are mentioned for dealing with them. However, the objections are serious enough, and the remedies partial enough, that it seems hardly warranted to call experiments the “gold standard” of causal in…
Sex, dependency, and helping
Demand characteristics and three conceptions of the frequently deceived subject
Generalizing Causal Knowledge in the Policy Sciences: External Validity as a Task of Both Multiattribute Representation and Multiattribute Extrapolation
Empirical tests of the absolute sleeper effect predicted from the discounting cue hypothesis
Evaluating the Rhetoric of Crisis: A Case Study of Criminal Victimization of the Elderly
This article begins by documenting claims that victimization of the elderly has recently reached crisis proportions. Four definitions of crisis are offered, and the social science evidence relating victimization to each of these definitions is presented. It does not seem (1) that the elderly are victimized more often than persons in other age groups, (2) that the rate of increase in their victimization is greater than for other age groups, or (3)…
Contemporary Thinking About Causation in Evaluation: A Dialogue With Tom Cook and Michael Scriven
Legitimate knowledge claims about causation have been a central concern among evaluators and applied researchers for several decades and often have been the subject of heated debates. In recent years these debates have resurfaced with a renewed intensity, due in part to the priority currently being given to randomized experiments by many funders of evaluation studies, such as the Institute for Educational Sciences. In this dialogue, which took pl…
Emergent Principles for the Design, Implementation, and Analysis of Cluster-Based Experiments in Social Science
In experimentally designed research, many good reasons exist for assigning groups or clusters to treatments rather than individuals. This article discusses them. But cluster-level designs face some unique or exacerbated challenges. The article identifies them and offers some principles about them. One emphasizes how statistical power and sample size estimation depend on intraclass correlations, particularly after conditioning on the use of cluste…
The causal assumptions of quasi-experimental practice: The origins of quasi-experimental practice
Persistence of attitude change as a function of conclusion reexposure: A laboratory-field experiment
Temporal mechanisms mediating attitude change after underpayment and overpayment
Competence, counterarguing, and attitude change
Cognitive, Behavioral and Temporal Effects of Confronting a Belief with Its Costly Action Implications
Thomas D. Cook, John R. Burd, Terence L. Talbert, Cognitive, Behavioral and Temporal Effects of Confronting a Belief with Its Costly Action Implications, Sociometry, Vol. 33, No. 3 (Sep., 1970), pp. 358-369
Demand characteristics and three conceptions of the frequently deceived subject
The effects of suspiciousness of deception and the perceived legitimacy of deception on task performance in an attitude change experiment 1
S ummary Subjects participated in two immediately consecutive experiments In the first, they either experienced a deception and debriefing, learned about deception in the abstract, did not learn about deception In the second, they either did or did not hear a reference to the possibility of a deception in that experiment A measure of incidental learning of the message in the second experiment showed that experiencing deception and learning about …
Sex, dependency, and helping
Attitude toward troop withdrawal from Indochina as a function of draft number: Dissonance or self-interest
Attitude change and the paired-associate learning of minimal cognitive elements 1
Subject effects in laboratory research: An examination of subject roles, demand characteristics, and valid inference
Vulnerability to Draft and Attitudes toward Troop Withdrawal from Indochina: Replication and Refinement
493 men recorded their opinions about when troops should be withdrawn from Indochina both before and after they were assigned a random draft lottery number. Persons assigned higher draft numbers which exempted them from military service advocated speedier withdrawal than did persons assigned numbers in the middle of the distribution. This effect replicated a finding from earlier research and was interpreted within a dissonance/equity framework. T…
The Educational Impact
Journal Article The Educational Impact Get access Thomas D. Cook, Thomas D. Cook 1Thomas D. Cook is Associate Professor of Psychology at Northwestern University Search for other works by this author on: Oxford Academic Google Scholar Ross F. Conner Ross F. Conner 2Ross F. Conner is Assistant Professor of Social Ecology the University of California at Irvine Search for other works by this author on: Oxford Academic Google Scholar Journal of Commun…
Evaluating the Rhetoric of Crisis: A Case Study of Criminal Victimization of the Elderly
This article begins by documenting claims that victimization of the elderly has recently reached crisis proportions. Four definitions of crisis are offered, and the social science evidence relating victimization to each of these definitions is presented. It does not seem (1) that the elderly are victimized more often than persons in other age groups, (2) that the rate of increase in their victimization is greater than for other age groups, or (3)…
"Sesame Street" Revisited
Academic and Entrepreneurial Research: The Consequences of Diversity in Federal Evaluation Studies. Ilene N. Bernstein , Howard E. Freeman
Empirical tests of the absolute sleeper effect predicted from the discounting cue hypothesis
Qualitative and quantitative methods in evaluation research
History of the sleeper effect: Some logical pitfalls in accepting the null hypothesis
Regression Analysis and Geographic Models: A Comment
History of the sleeper effect: Some logical pitfalls in accepting the null hypothesis
Equity theory and the cognitive ability of children
Evaluation Studies: Review Annual Volume 3, 1978
The Misutilization of Evaluation Research: Some Pitfalls of Definition
What differentiates meta-analysis from other forms of review1
An issue of friendly disagreement between Cooper and Arkin (1981) and Cook and Leviton (1980) involves the definition of meta-analysis. Cooper and Arkin favor a definition stressing the degree of quantification in review studies, while we claim that studies labeled as “meta-analysis” are differentiated from other reviews in terms of the aggregation of effect sizes or probability values across studies for the purpose of reaching numerical estimate…
Time-Series Modelling in a Regional Economy: An Exposition of Box-Jenkins Techniques
Box-Jenkins techniques are shown to be a useful tool for analyzing regional economic activity. This paper identifies temporal, spatial, and causal properties derived from these techniques. A case study is presented illustrating the univariate, transfer-function, multivariate, and multivariate transfer-function methods with household formation and employment time-series data
Psychology (53 works) · Computer Science (39 works) · Mathematics (26 works) · Sociology (26 works) · Social Psychology (23 works) · Statistics (22 works) · Econometrics (19 works) · Advanced Causal Inference Techniques (18 works) · Political science (15 works) · Developmental psychology (11 works)