Carlisle Rainey
Biographic Data
| ID | 1102978 |
|---|---|
| NAME | Carlisle Rainey |
| GIVEN NAMES | Carlisle |
| FAMILY NAME | Rainey |
| SIGNATURE | RAINEY C |
| AFFILIATIONS | Florida State University |
| ORCID | 0000-0002-8728-4696 |
| VERIFIED | Yes |
| TOTAL WORKS | 21 |
| TOTAL CITATIONS | 480 |
| AUTHOR COUNT | 21 |
| EDITOR COUNT | 0 |
| FIRST PUBLICATION YEAR | 2014 |
| LATEST PUBLICATION YEAR | 2025 |
| H-INDEX | 10 |
Data and Code Availability in Political Science Publications from 1995 to 2022 – Addendum
its figures and tables, but there are substantively small differences between the replication and the printed results
The Limits (and Strengths) of Single-Topic Experiments
We examine the generalizability of single-topic studies, focusing on how often their confidence intervals capture the typical treatment effect from a larger population of possible studies. We show that the confidence intervals from these single-topic studies capture the typical effect from a population of topics at well below the nominal rate. For a plausible scenario, the confidence interval from a single-topic study might only be half as wide a…
Data and Code Availability in Political Science Publications from 1995 to 2022
In this article, we assess the availability of reproduction archives in political science. By “reproduction archive,” we mean the data and code supporting quantitative research articles that allows others to reproduce the computations described in the published article. We collect a random sample of quantitative research articles published in political science from 1995 to 2022. We find that—even in 2022—most quantitative research articles do not…
A careful consideration of CLARIFY: Simulation-Induced Bias in Point Estimates of Quantities of Interest
Some work in political methodology recommends that applied researchers obtain point estimates of quantities of interest by simulating model coefficients, transforming these simulated coefficients into simulated quantities of interest, and then averaging the simulated quantities of interest (e.g., CLARIFY). But other work advises applied researchers to directly transform coefficient estimates to estimate quantities of interest. I point out that th…
A careful consideration of CLARIFY: Simulation-Induced Bias in Point Estimates of Quantities of Interest – Corrigendum
Generalizing Survey Experiments Using Topic Sampling: An Application to Party Cues
Hypothesis Tests under Separation
Separation commonly occurs in political science, usually when a binary explanatory variable perfectly predicts a binary outcome. In these situations, methodologists often recommend penalized maximum likelihood or Bayesian estimation. But researchers might struggle to identify an appropriate penalty or prior distribution. Fortunately, I show that researchers can easily test hypotheses about the model coefficients with standard frequentist tools. W…
Estimators for Topic-Sampling Designs
When researchers design an experiment, they usually hold potentially relevant features of the experiment constant. We call these details the “topic” of the experiment. For example, researchers studying the impact of party cues on attitudes must inform respondents of the parties’ positions on a particular policy . In doing so, researchers implement just one of many possible designs . Clifford, Leeper, and Rainey (2023. “Generalizing Survey Experim…
Estimating logit models with small samples
In small samples, maximum likelihood (ML) estimates of logit model coefficients have substantial bias away from zero. As a solution, we remind political scientists of Firth's (1993,Biometrika,80, 27–38) penalized maximum likelihood (PML) estimator. Prior research has described and used PML, especially in the context of separation, but its small sample properties remain under-appreciated. The PML estimator eliminates most of the bias and, perhaps …
When BLUE is not best: Non-Normal Errors and the Linear Model
Researchers in political science often estimate linear models of continuous outcomes using least squares. While it is well known that least-squares estimates are sensitive to single, unusual data points, this knowledge has not led to careful practices when using least-squares estimators. Using statistical theory and Monte Carlo simulations, we highlight the importance of using more robust estimators along with variable transformations. We also di…
Unreliable Inferences About Unobserved Processes: A Critique of Partial Observability Models
Methodologists and econometricians advocate the partial observability model as a tool that enables researchers to estimate the distinct effects of a single explanatory variable on two partially observable outcome variables. However, we show that when the explanatory variable of interest influences both partially observable outcomes, the partial observability model estimates are extremely sensitive to misspecification. We use Monte Carlo simulatio…
Transformation-Induced Bias: Unbiased Coefficients Do Not Imply Unbiased Quantities of Interest
Political scientists commonly focus on quantities of interest computed from model coefficients rather than on the coefficients themselves. However, the quantities of interest, such as predicted probabilities, first differences, and marginal effects, do not necessarily inherit the small-sample properties of the coefficient estimates. Indeed, unbiased coefficient estimates are neither necessary nor sufficient for unbiased estimates of the quantitie…
Does district magnitude matter? The case of Taiwan
Compression and Conditional Effects: A Product Term Is Essential When Using Logistic Regression to Test for Interaction
Previous research in political methodology argues that researchers do not need to include a product term in a logistic regression model to test for interaction if they suspect interaction due to compression alone. I disagree with this claim and offer analytical arguments and simulation evidence that when researchers incorrectly theorize interaction due to compression, models without a product term bias the researcher, sometimes heavily, toward fi…
Dealing with Separation in Logistic Regression Models
When facing small numbers of observations or rare events, political scientists often encounter separation, in which explanatory variables perfectly predict binary events or nonevents. In this situation, maximum likelihood provides implausible estimates and the researcher might want incorporate some form of prior information into the model. The most sophisticated research uses Jeffreys’ invariant prior to stabilize the estimates. While Jeffreys’ p…
Moral Concerns and Policy Attitudes: Investigating the Influence of Elite Rhetoric
A growing body of research documents the crucial role played by moral concerns in the formation of attitudes and a wide range of political behaviors. Yet extant models of moral judgment portray a direct linkage between moral intuitions and policy attitudes, leaving little room for the influence of political context. In this article, we argue that political rhetoric plays an important role in facilitating the connection between moral intuitions an…
Substantive Importance and the Veil of Statistical Significance
Political science is gradually moving away from an exclusive focus on statistical significance and toward an emphasis on the magnitude and importance of effects. While we welcome this change, we argue that the current practice of “magnitude-and-significance,” in which researchers only interpret the magnitude of a statistically significant point estimate, barely improves the much-maligned “sign-and-significance” approach, in which researchers focu…
Strategic mobilization: Why proportional representation decreases voter mobilization
The Politics of Need: Examining Governors' Decisions to Oppose the “Obamacare” Medicaid Expansion
This article explains governors' decisions to support or oppose Medicaid expansions offered under the 2010 Patient Protection and Affordable Care Act. We theorize that governors' decisions to oppose the funding should depend on both political demands and the level of need in the state, though politics and need are often in tension. We find that governors' partisanship and the composition of the legislature have substantively meaningful effects on…
Arguing for a Negligible Effect
Political scientists often theorize that an explanatory variable should have “no effect” and support this claim by demonstrating that its coefficient's estimate is not statistically significant. This empirical argument is quite weak, but I introduce applied researchers to simple, powerful tools that can strengthen their arguments for this hypothesis. With several supporting examples, I illustrate that researchers can use 90% confidence intervals …
The Question(s) of Political Knowledge
Political knowledge is a central concept in the study of public opinion and political behavior. Yet what the field collectively believes about this construct is based on dozens of studies using different indicators of knowledge. We identify two theoretically relevant dimensions: atemporaldimension that corresponds to the time when a fact was established and atopicaldimension that relates to whether the fact is policy-specific or general. The resu…
Arguing for a Negligible Effect
Political scientists often theorize that an explanatory variable should have “no effect” and support this claim by demonstrating that its coefficient's estimate is not statistically significant. This empirical argument is quite weak, but I introduce applied researchers to simple, powerful tools that can strengthen their arguments for this hypothesis. With several supporting examples, I illustrate that researchers can use 90% confidence intervals …
The Question(s) of Political Knowledge
Political knowledge is a central concept in the study of public opinion and political behavior. Yet what the field collectively believes about this construct is based on dozens of studies using different indicators of knowledge. We identify two theoretically relevant dimensions: atemporaldimension that corresponds to the time when a fact was established and atopicaldimension that relates to whether the fact is policy-specific or general. The resu…
Moral Concerns and Policy Attitudes: Investigating the Influence of Elite Rhetoric
A growing body of research documents the crucial role played by moral concerns in the formation of attitudes and a wide range of political behaviors. Yet extant models of moral judgment portray a direct linkage between moral intuitions and policy attitudes, leaving little room for the influence of political context. In this article, we argue that political rhetoric plays an important role in facilitating the connection between moral intuitions an…
The Politics of Need: Examining Governors' Decisions to Oppose the “Obamacare” Medicaid Expansion
This article explains governors' decisions to support or oppose Medicaid expansions offered under the 2010 Patient Protection and Affordable Care Act. We theorize that governors' decisions to oppose the funding should depend on both political demands and the level of need in the state, though politics and need are often in tension. We find that governors' partisanship and the composition of the legislature have substantively meaningful effects on…
Estimating logit models with small samples
In small samples, maximum likelihood (ML) estimates of logit model coefficients have substantial bias away from zero. As a solution, we remind political scientists of Firth's (1993,Biometrika,80, 27–38) penalized maximum likelihood (PML) estimator. Prior research has described and used PML, especially in the context of separation, but its small sample properties remain under-appreciated. The PML estimator eliminates most of the bias and, perhaps …
Compression and Conditional Effects: A Product Term Is Essential When Using Logistic Regression to Test for Interaction
Previous research in political methodology argues that researchers do not need to include a product term in a logistic regression model to test for interaction if they suspect interaction due to compression alone. I disagree with this claim and offer analytical arguments and simulation evidence that when researchers incorrectly theorize interaction due to compression, models without a product term bias the researcher, sometimes heavily, toward fi…
Dealing with Separation in Logistic Regression Models
When facing small numbers of observations or rare events, political scientists often encounter separation, in which explanatory variables perfectly predict binary events or nonevents. In this situation, maximum likelihood provides implausible estimates and the researcher might want incorporate some form of prior information into the model. The most sophisticated research uses Jeffreys’ invariant prior to stabilize the estimates. While Jeffreys’ p…
Generalizing Survey Experiments Using Topic Sampling: An Application to Party Cues
Substantive Importance and the Veil of Statistical Significance
Political science is gradually moving away from an exclusive focus on statistical significance and toward an emphasis on the magnitude and importance of effects. While we welcome this change, we argue that the current practice of “magnitude-and-significance,” in which researchers only interpret the magnitude of a statistically significant point estimate, barely improves the much-maligned “sign-and-significance” approach, in which researchers focu…
Strategic mobilization: Why proportional representation decreases voter mobilization
Transformation-Induced Bias: Unbiased Coefficients Do Not Imply Unbiased Quantities of Interest
Political scientists commonly focus on quantities of interest computed from model coefficients rather than on the coefficients themselves. However, the quantities of interest, such as predicted probabilities, first differences, and marginal effects, do not necessarily inherit the small-sample properties of the coefficient estimates. Indeed, unbiased coefficient estimates are neither necessary nor sufficient for unbiased estimates of the quantitie…
Estimators for Topic-Sampling Designs
When researchers design an experiment, they usually hold potentially relevant features of the experiment constant. We call these details the “topic” of the experiment. For example, researchers studying the impact of party cues on attitudes must inform respondents of the parties’ positions on a particular policy . In doing so, researchers implement just one of many possible designs . Clifford, Leeper, and Rainey (2023. “Generalizing Survey Experim…
The Limits (and Strengths) of Single-Topic Experiments
We examine the generalizability of single-topic studies, focusing on how often their confidence intervals capture the typical treatment effect from a larger population of possible studies. We show that the confidence intervals from these single-topic studies capture the typical effect from a population of topics at well below the nominal rate. For a plausible scenario, the confidence interval from a single-topic study might only be half as wide a…
When BLUE is not best: Non-Normal Errors and the Linear Model
Researchers in political science often estimate linear models of continuous outcomes using least squares. While it is well known that least-squares estimates are sensitive to single, unusual data points, this knowledge has not led to careful practices when using least-squares estimators. Using statistical theory and Monte Carlo simulations, we highlight the importance of using more robust estimators along with variable transformations. We also di…
Data and Code Availability in Political Science Publications from 1995 to 2022
In this article, we assess the availability of reproduction archives in political science. By “reproduction archive,” we mean the data and code supporting quantitative research articles that allows others to reproduce the computations described in the published article. We collect a random sample of quantitative research articles published in political science from 1995 to 2022. We find that—even in 2022—most quantitative research articles do not…
A careful consideration of CLARIFY: Simulation-Induced Bias in Point Estimates of Quantities of Interest
Some work in political methodology recommends that applied researchers obtain point estimates of quantities of interest by simulating model coefficients, transforming these simulated coefficients into simulated quantities of interest, and then averaging the simulated quantities of interest (e.g., CLARIFY). But other work advises applied researchers to directly transform coefficient estimates to estimate quantities of interest. I point out that th…
Hypothesis Tests under Separation
Separation commonly occurs in political science, usually when a binary explanatory variable perfectly predicts a binary outcome. In these situations, methodologists often recommend penalized maximum likelihood or Bayesian estimation. But researchers might struggle to identify an appropriate penalty or prior distribution. Fortunately, I show that researchers can easily test hypotheses about the model coefficients with standard frequentist tools. W…
Unreliable Inferences About Unobserved Processes: A Critique of Partial Observability Models
Methodologists and econometricians advocate the partial observability model as a tool that enables researchers to estimate the distinct effects of a single explanatory variable on two partially observable outcome variables. However, we show that when the explanatory variable of interest influences both partially observable outcomes, the partial observability model estimates are extremely sensitive to misspecification. We use Monte Carlo simulatio…
The Politics of Need: Examining Governors' Decisions to Oppose the “Obamacare” Medicaid Expansion
This article explains governors' decisions to support or oppose Medicaid expansions offered under the 2010 Patient Protection and Affordable Care Act. We theorize that governors' decisions to oppose the funding should depend on both political demands and the level of need in the state, though politics and need are often in tension. We find that governors' partisanship and the composition of the legislature have substantively meaningful effects on…
Arguing for a Negligible Effect
Political scientists often theorize that an explanatory variable should have “no effect” and support this claim by demonstrating that its coefficient's estimate is not statistically significant. This empirical argument is quite weak, but I introduce applied researchers to simple, powerful tools that can strengthen their arguments for this hypothesis. With several supporting examples, I illustrate that researchers can use 90% confidence intervals …
The Question(s) of Political Knowledge
Political knowledge is a central concept in the study of public opinion and political behavior. Yet what the field collectively believes about this construct is based on dozens of studies using different indicators of knowledge. We identify two theoretically relevant dimensions: atemporaldimension that corresponds to the time when a fact was established and atopicaldimension that relates to whether the fact is policy-specific or general. The resu…
Moral Concerns and Policy Attitudes: Investigating the Influence of Elite Rhetoric
A growing body of research documents the crucial role played by moral concerns in the formation of attitudes and a wide range of political behaviors. Yet extant models of moral judgment portray a direct linkage between moral intuitions and policy attitudes, leaving little room for the influence of political context. In this article, we argue that political rhetoric plays an important role in facilitating the connection between moral intuitions an…
Substantive Importance and the Veil of Statistical Significance
Political science is gradually moving away from an exclusive focus on statistical significance and toward an emphasis on the magnitude and importance of effects. While we welcome this change, we argue that the current practice of “magnitude-and-significance,” in which researchers only interpret the magnitude of a statistically significant point estimate, barely improves the much-maligned “sign-and-significance” approach, in which researchers focu…
Strategic mobilization: Why proportional representation decreases voter mobilization
Does district magnitude matter? The case of Taiwan
Compression and Conditional Effects: A Product Term Is Essential When Using Logistic Regression to Test for Interaction
Previous research in political methodology argues that researchers do not need to include a product term in a logistic regression model to test for interaction if they suspect interaction due to compression alone. I disagree with this claim and offer analytical arguments and simulation evidence that when researchers incorrectly theorize interaction due to compression, models without a product term bias the researcher, sometimes heavily, toward fi…
Dealing with Separation in Logistic Regression Models
When facing small numbers of observations or rare events, political scientists often encounter separation, in which explanatory variables perfectly predict binary events or nonevents. In this situation, maximum likelihood provides implausible estimates and the researcher might want incorporate some form of prior information into the model. The most sophisticated research uses Jeffreys’ invariant prior to stabilize the estimates. While Jeffreys’ p…
Transformation-Induced Bias: Unbiased Coefficients Do Not Imply Unbiased Quantities of Interest
Political scientists commonly focus on quantities of interest computed from model coefficients rather than on the coefficients themselves. However, the quantities of interest, such as predicted probabilities, first differences, and marginal effects, do not necessarily inherit the small-sample properties of the coefficient estimates. Indeed, unbiased coefficient estimates are neither necessary nor sufficient for unbiased estimates of the quantitie…
When BLUE is not best: Non-Normal Errors and the Linear Model
Researchers in political science often estimate linear models of continuous outcomes using least squares. While it is well known that least-squares estimates are sensitive to single, unusual data points, this knowledge has not led to careful practices when using least-squares estimators. Using statistical theory and Monte Carlo simulations, we highlight the importance of using more robust estimators along with variable transformations. We also di…
Unreliable Inferences About Unobserved Processes: A Critique of Partial Observability Models
Methodologists and econometricians advocate the partial observability model as a tool that enables researchers to estimate the distinct effects of a single explanatory variable on two partially observable outcome variables. However, we show that when the explanatory variable of interest influences both partially observable outcomes, the partial observability model estimates are extremely sensitive to misspecification. We use Monte Carlo simulatio…
Estimating logit models with small samples
In small samples, maximum likelihood (ML) estimates of logit model coefficients have substantial bias away from zero. As a solution, we remind political scientists of Firth's (1993,Biometrika,80, 27–38) penalized maximum likelihood (PML) estimator. Prior research has described and used PML, especially in the context of separation, but its small sample properties remain under-appreciated. The PML estimator eliminates most of the bias and, perhaps …
A careful consideration of CLARIFY: Simulation-Induced Bias in Point Estimates of Quantities of Interest
Some work in political methodology recommends that applied researchers obtain point estimates of quantities of interest by simulating model coefficients, transforming these simulated coefficients into simulated quantities of interest, and then averaging the simulated quantities of interest (e.g., CLARIFY). But other work advises applied researchers to directly transform coefficient estimates to estimate quantities of interest. I point out that th…
A careful consideration of CLARIFY: Simulation-Induced Bias in Point Estimates of Quantities of Interest – Corrigendum
Generalizing Survey Experiments Using Topic Sampling: An Application to Party Cues
Hypothesis Tests under Separation
Separation commonly occurs in political science, usually when a binary explanatory variable perfectly predicts a binary outcome. In these situations, methodologists often recommend penalized maximum likelihood or Bayesian estimation. But researchers might struggle to identify an appropriate penalty or prior distribution. Fortunately, I show that researchers can easily test hypotheses about the model coefficients with standard frequentist tools. W…
Estimators for Topic-Sampling Designs
When researchers design an experiment, they usually hold potentially relevant features of the experiment constant. We call these details the “topic” of the experiment. For example, researchers studying the impact of party cues on attitudes must inform respondents of the parties’ positions on a particular policy . In doing so, researchers implement just one of many possible designs . Clifford, Leeper, and Rainey (2023. “Generalizing Survey Experim…
Data and Code Availability in Political Science Publications from 1995 to 2022 – Addendum
its figures and tables, but there are substantively small differences between the replication and the printed results
The Limits (and Strengths) of Single-Topic Experiments
We examine the generalizability of single-topic studies, focusing on how often their confidence intervals capture the typical treatment effect from a larger population of possible studies. We show that the confidence intervals from these single-topic studies capture the typical effect from a population of topics at well below the nominal rate. For a plausible scenario, the confidence interval from a single-topic study might only be half as wide a…
Data and Code Availability in Political Science Publications from 1995 to 2022
In this article, we assess the availability of reproduction archives in political science. By “reproduction archive,” we mean the data and code supporting quantitative research articles that allows others to reproduce the computations described in the published article. We collect a random sample of quantitative research articles published in political science from 1995 to 2022. We find that—even in 2022—most quantitative research articles do not…
Mathematics (13 works) · Electoral Systems and Political Participation (12 works) · Econometrics (11 works) · Statistics (11 works) · Computer Science (10 works) · Qualitative Comparative Analysis Research (9 works) · Political science (8 works) · Politics (8 works) · Economics (7 works) · Law (7 works)