Donald W Zimmerman
Biographic Data
| ID | 265849 |
|---|---|
| NAME | Donald W Zimmerman |
| GIVEN NAMES | Donald W |
| FAMILY NAME | Zimmerman |
| SIGNATURE | ZIMMERMAN D W |
| AFFILIATIONS | Carleton University |
| VERIFIED | No |
| TOTAL WORKS | 44 |
| TOTAL CITATIONS | 9 |
| WORKS AS AUTHOR | 44 |
| WORKS AS EDITOR | 0 |
| FIRST PUBLICATION YEAR | 1963 |
| LATEST PUBLICATION YEAR | 2012 |
| H-INDEX | 2 |
A Note on the Dependence of Heritability on Variances of Genetic and Environmental Components
A Note on the Dependence of Heritability on Variances of Genetic and Environmental Components Donald W. Zimmerman The mathematics of population genetics is highly developed and has led to many important insights. Nevertheless, it is possible that some simple algebraic expressions, not widely known, can enable researchers to view existing findings from a somewhat different perspective. These simple derivations perhaps also could have benefit in ed…
Can Percentiles Replace Raw Scores in the Statistical Analysis of Test Data
Educational and psychological testing textbooks typically warn of the inappropriateness of performing arithmetic operations and statistical analysis on percentiles instead of raw scores. This seems inconsistent with the well-established finding that transforming scores to ranks and using nonparametric methods often improves the validity and power of significance tests for nonnormal distributions. This study compared Student’s t test performed on …
Statistical Significance Levels of Nonparametric Tests Biased by Heterogeneous Variances of Treatment Groups
The statistical significance levels of the Wilcoxon-Mann-Whitney test and the Kruskal-Wallis test are substantially biased by heterogeneous variances of treatment groups--even when sample sizes are equal. Under these conditions, the Type I error probabilities of the nonparametric tests, performed at the .01, .05, and .10 significance levels, increase by as much as 40%-50% in many cases and sometimes as much as 300%. The bias increases systematica…
Commentary on ‘Science, Measurement, and Validity
How Should Classical Test Theory Have Defined Validity
Some Properties of Preliminary Tests of Equality of Variances in the Two-Sample Location Problem
A simulation study was conducted to examine probabilities of Type I errors of the two-sample Student t test, the Wilcoxon–Mann–Whitney test, and the Welch separate-variances t test under violation of homogeneity of variance. Two-stage procedures in which the choice of a significance test in the second stage is determined by the outcome of a preliminary test of equality of variances in the first stage were also examined. Type I error rates of both…
An Efficient Alternative to the Wilcoxon Signed-Ranks Test for Paired Nonnormal Data
The Wilcoxon signed-ranks test is widely used to analyze paired data when the normality assumption is doubtful and the paired-samples t test is inappropriate. An alternative procedure was investigated: It performs an independent-samples Student t test on ranks replacing the initial scores and corrects the t statistic by a function of the sample correlation coefficient. The procedure maintains the significance level for normal and eight kinds of n…
Increasing the Power of the Anova F Test for Outlier-Prone Distributions by Modified Ranking Methods
A simulation study performed analysis of variance (ANOVA) F tests on samples from two outlier-resistant densities, four outlier-prone densities, and four truncated forms of the same outlier-prone densities. The tests were repeated on the ranks of the sample values and on scores resulting from four transformations, in which ranks were replaced by a smaller set of integers or by numbers that included a random component. Modified or “distorted” rank…
A Note on the Influence of Outliers on Parametric and Nonparametric Tests
Extremely deviant scores, or outliers, reduce the probability of Type I errors of the Student t test and, at the same time, substantially increase the probability of Type II errors, so that power declines. The magnitude of the change depends jointly on the probability of occurrence of an outlier and its extremity, or its distance from the mean. Although outliers do not modify the probability of Type I errors of the Mann-Whitney-Wilcoxon test, the…
A Note on the F Test for Equal Variances under Violation of Random Sampling
This study examined the influence of nonindependence of within-group sample values on the F test for equal variances. The probabilities of Type I and Type II errors were altered substantially when the correlation between successive sample values in one group differed from the correlation in the other group, but they were not affected when the two correlations were equal. A modified F statistic obtained from variance estimates that were corrected …
Coefficient Alpha as an Estimate of Test Reliability Under Violation of Two Assumptions
Through use of computer simulation, the central tendency and variability of coefficient alpha were examined under violation of two assumptions made in the derivation of the formula. When assumptions were satisfied, the mean value of coefficient alpha was extremely close to the population reliability coefficient, but values were highly variable. This result was independent of the shape of the population distribution of test scores. Coefficient alp…
Significance Testing of Correlation Using Scores, Ranks, and Modified Ranks
A computer simulation study compared significance tests of correlation coefficients calculated from initial scores, from ranks assigned by the Spearman method, and from three kinds of modified ranks in which N sample values were replaced by N12, N/3, and N14 integers. Tests based on the initial scores are more powerful than those based on the various ranks for normal distributions, whereas the reverse is true for mixed-normal, exponential, and Ca…
Mimicking Properties of Nonparametric Rank Tests Using Scores that are not Ranks
In a computer simulation study, random samples from a uniform density were substituted for each of two independent samples from normal and various nonnormal densities. This procedure was compared with conventional ranking and with Bell and Doksum's (1965) procedure, which substituted random normal deviates for initial sample values. After performing the Student t test, the program transformed the initial scores and performed additional t tests on…
Increasing the Power of Tests of Location and Correlation by Transforming Scores to Ranks and Indicators
For various nonnormal distributions, the power of the Student t test can be increased if continuous measures are transformed to ranks before the test is performed. The power of the test can also be increased almost as much and, even more for some distributions, if measures are replaced by dichotomous variables with the values 0 and 1, instead of ranks. Similarly, the power of a significance test of correlation can be increased if scores are trans…
The Relative Power of the Wilcoxon-Mann-Whitney Test and Student t Test Under Simple Bounded Transformations
The Wilcoxon-Mann-Whitney test and related nonparametric statistical methods based on ranks have been found to be more powerful than parametric tests such as the Student t test for various heavy-tailed distributions, including the exponential, Cauchy, and mixed-normal distributions, both in the asymptotic limit and for relatively small sample sizes. The present computer simulation study provided evidence that this power advantage results from red…
Statistical Power Analysis and Reliability of Measurement
Reliability of measurement as a determinant of the power of significance tests is investigated using concepts from statistical power analysis and from test and measurement theory. Results of specific power calculations based on measures of effect size together with particular reliability coefficients are presented in the form of tables. The importance of assumptions about true variance, error variance, and observed variance in determining the rel…
Note on the reliability of experimental measures and the power of significance tests
Note on the reliability of experimental measures and the power of significance tests
Variability of Deviation IQ's Based on Multiple Choice Test Scores
A computer program simulated guessing on multiple-choice test items and calculated deviation IQ's from observed scores which contained a guessing component. The entire procedure was replicated 5000 times for each of a variety of assumptions about the number of test items, the number of choices per item, and the distribution of the number of items "known" by examinees before guessing. Extensive variability in deviation IQ's due entirely to chance …
Elimination of Bias in Test Scores
This paper demonstrates that correlation between bias components of test scores and unbiased observed scores is an effective predictor of changes in reliability and validity resulting from elimination of bias. Plausible assumptions about the value of this correlation and the sizes of related variance components yield the conclusion that reduction in reliability and validity is a possible outcome. This reduction could result in marginally valid or…
The Mathematical Definition of Test Validity
A definition of test validity as the ratio of a covariance term to a variance term, analogous to the classical definition of test reliability as the proportion of observed variance which is true variance, is proposed. This definition leads to simple validity formulas which do not depend on an assumption of experimental independence. When error scores on distinct tests are uncorrelated, the proposed definition coincides with the usual one, but it …
On the High Predictive Potential of Change and Growth Measures
Formulas in the statistical theory of test scores have led some psychometricians to believe that measures of change and growth have questionable value in research. However, certain combinations of parameters, when substituted into these formulas, yield reliable change scores and high non-spurious correlations between change scores and independent criterion scores, even when pretest scores are not good predictors of either changes or the criterion…
Comparative Validity of Simple and Residualized Difference Scores
Apart from some anomalous exceptions which are not likely to occur in testing practice, the predictor-criterion validity of simple difference scores is greater than, less than, or equal to that of residualized difference scores depending on whether the correlation between observed pretest and posttest scores is greater than, less than, or equal to the ratio of the standard deviations of pretest and posttest scores
Element of Chance and Comparative Reliability of Matching Tests and Multiple-Choice Tests
Simple formulas for the reliability of matching tests and multiple-choice tests, based on the assumption that chance success by guessing is the only source of error variation, are derived. The reliabilities of the two types of tests are compared, and an explicit formula relating their reliability coefficients when relevant test parameters remain fixed is presented. It is found that, in the majority of cases which are likely in practice, matching …
A Note on the Correlation of Gains and Initial Status
By writing the familiar equation for the correlation of gains and initial status in a form which reveals that the correlation does not depend on the standard deviations of pretest and posttest scores separately, but only on their ratio, it is possible to exhibit a one-parameter family of functions which throws new light on this relationship. The same family of functions provides information about the correlation between the true components of gai…
A Note on the Influence of Outliers on Parametric and Nonparametric Tests
Extremely deviant scores, or outliers, reduce the probability of Type I errors of the Student t test and, at the same time, substantially increase the probability of Type II errors, so that power declines. The magnitude of the change depends jointly on the probability of occurrence of an outlier and its extremity, or its distance from the mean. Although outliers do not modify the probability of Type I errors of the Mann-Whitney-Wilcoxon test, the…
Statistical Power Analysis and Reliability of Measurement
Reliability of measurement as a determinant of the power of significance tests is investigated using concepts from statistical power analysis and from test and measurement theory. Results of specific power calculations based on measures of effect size together with particular reliability coefficients are presented in the form of tables. The importance of assumptions about true variance, error variance, and observed variance in determining the rel…
Statistical Significance Levels of Nonparametric Tests Biased by Heterogeneous Variances of Treatment Groups
The statistical significance levels of the Wilcoxon-Mann-Whitney test and the Kruskal-Wallis test are substantially biased by heterogeneous variances of treatment groups--even when sample sizes are equal. Under these conditions, the Type I error probabilities of the nonparametric tests, performed at the .01, .05, and .10 significance levels, increase by as much as 40%-50% in many cases and sometimes as much as 300%. The bias increases systematica…
Some Properties of Preliminary Tests of Equality of Variances in the Two-Sample Location Problem
A simulation study was conducted to examine probabilities of Type I errors of the two-sample Student t test, the Wilcoxon–Mann–Whitney test, and the Welch separate-variances t test under violation of homogeneity of variance. Two-stage procedures in which the choice of a significance test in the second stage is determined by the outcome of a preliminary test of equality of variances in the first stage were also examined. Type I error rates of both…
An Efficient Alternative to the Wilcoxon Signed-Ranks Test for Paired Nonnormal Data
The Wilcoxon signed-ranks test is widely used to analyze paired data when the normality assumption is doubtful and the paired-samples t test is inappropriate. An alternative procedure was investigated: It performs an independent-samples Student t test on ranks replacing the initial scores and corrects the t statistic by a function of the sample correlation coefficient. The procedure maintains the significance level for normal and eight kinds of n…
Influence of Conditions at the Time of Reinforcement upon the Strength of a Secondary Reinforcement Effect
Animals were trained to run through a guillotine door into an adjacent compartment for water reinforcement. They were then given access to the compartment as a consequence of bar-pressing, but no water. At the time of testing for bar-pressing various changes were made in the type of response required. For other groups of animals variations were made in the relative sizes of the two compartments during both training and testing. It was found that …
Influence of Three Stimulus Conditions upon the Strength of a Secondary Reinforcement Effect
Animals were trained to approach a water dipper at the onset of a light and a tone, water being received following approach on an intermittent schedule. For three groups of animals bar-pressing was then reinforced on an intermittent schedule, using three stimulus conditions: light and tone only; light, tone, and dry dipper; and light, tone, and interrupted presentation of dry dipper. The interrupted presentation proved to be the most effective re…
Chance Success Due to Guessing and Non-Independence of True Scores and Error Scores in Multiple-Choice Tests
The effect of chance success due to guessing upon the variance of multiple-choice test scores was estimated from prepared distributions of large numbers of scores. Each score consisted of an assumed “true score” component and an “error score” component generated by a computer. A large negative correlation was found between true scores and error scores and a positive correlation between error scores and error scores. The equation showing reliabili…
Effect of Chance Success Due to Guessing on Error of Measurement in Multiple-Choice Tests
Chance success due to guessing is treated as a component of the error variance of a multiple-choice test score. It is shown that for a test of given item structure the minimum standard error of measurement can be estimated by the formula ( N − X ) / a . where N is the total number of items, X is the score, and a is the number of alternative choices per item. The significance of non-independence of true score and this component of error score on m…
Dependence of Reliability of Multiple-Choice Tests Upon Number of Choices per Item
An equation is derived which expresses test reliability as a function of number of item alternatives for the case in which only error due to guessing is present. This result is compared with the modified Spearman-Brown equation given by H. H. Remmers and his associates. Reliability coefficients predicted by these equations are compared with coefficients generated by a computer simulation method
Empirical Estimates of Intercorrelations among the Components of Scores on Multiple-Choice Tests
College students were instructed to indicate on various multiple-choice tests whether they “knew the answer” or “guessed” each item, and the results were treated as estimated true and error components of scores. The values of the intercorrelations of these components were similar to those given by a computer program described previously. The values found for all tests were consistent with the assumption that test scores consist of both independen…
Interpretation of the Standard Error of Measurement when True Scores and Error Scores on Mental Tests are Not Independent
It is shown that for the case of non-independence of true scores and error scores interpretation of the standard error of measurement is modified in two ways. First, the standard deviation of the distribution of error scores is given by a modified equation. Second, the confidence interval for true score varies with the individual's observed score. It is shown that the equation, s o = √ [ ( N − O / a ] + [ s o 2 ( r o o ̄ − r o o ) / r o o ̄ ] ̄ ,…
Dependence of Test Reliability Upon Heterogeneity of Individual and Group Score Distributions
An Item Sampling Model for the Reliability of Composite Tests
A Simplified Probability Model of Error of Measurement
A model of variability in measurement which does not employ the concepts of “true score” and “error score” is presented. Reference to an observed score random variable, X , together with the usual axioms of probability, is shown to be a satisfactory basis for derivation of results of the classical test theory which relate observable quantities. In addition, reliability formulas such as the KR 20 and KR 21 are obtained by construction of the obser…
Estimation of the Reliability of Composite Tests by a Computer Simulation Method
A computer program simulated variability in test scores by generating the values of random variables, X ij , each having possible values 1 and 0 with probability p ij and 1 — p ij , where the subscript i refers to items and the subscript j to persons. A matrix of p ij values was considered to be a representation of a given test. Reliability was interpreted as a measure of the consistency of X ij values over replications of the procedure and was e…
Effect of Variation in Probability of Guessing Correctly on Reliability of Multiple-Choice Tests
Variability of Test Scores and the Split-Half Reliability Coefficient
Probability Spaces and the Theory of Error of Measurement
A model of variability in measurement, which is sufficiently general for a variety of applications and which includes the main content of traditional theories of error of measurement and psychological tests, can be derived from the axioms of probability, without introducing “true values” and “errors.” Beginning with probability spaces (Ω, P 1 ) and (φ, P 2 ), the set Ω representing the outcomes of a measurement procedure and the set * representin…
Test Reliability and the Kuder-Richardson Formulas
Error and Reliability in Stochastic Processes and Psychological Measurement
The concepts of random error and reliability of measurements that are familiar in traditional theories based on the notions of “true values” and “errors” can be represented by a probability model having a simpler formal structure and fewer special assumptions about random sampling and independence of measurements. In this model formulas that relate observable events are derived from probability axioms and from primitive terms that refer to observ…
Two Concepts of “True Score” in Test Theory
Two concepts of “true score” in test theory are examined. Under one concept, the true score is identified with the expected value of the observed score, and it follows that reliability is the ratio of true variance to observed variance. Under the other concept, the true score is a constant which is not necessarily equal to the expected value of the observed score, and it follows that reliability is not necessarily equal to the ratio of true varia…
Test Theory With Minimal Assumptions
Using the concepts of conditional probability, conditional expectation, and conditional independence, the main results of the classical test theory model can be derived in a very few steps with minimal assumptions. It is well known that the variance of a random variable is the sum of the variance of its conditional expectation and the expectation of its conditional variance. These concepts lead to another fundamental relation: the reliability coe…
The Reliability of Difference Scores When Errors Are Correlated
The usual formulas for the reliability of differences between two test scores X and Y are based on the assumption that the error scores E X and E Y are uncorrelated. In modern developments of test score theory, such as that of Lord and Novick, a true score is defined as the expected value of an individual's observed score. This definition implies that true scores on any test are uncorrelated with error scores on any test, but it does not imply th…
On the High Predictive Potential of Change and Growth Measures
Formulas in the statistical theory of test scores have led some psychometricians to believe that measures of change and growth have questionable value in research. However, certain combinations of parameters, when substituted into these formulas, yield reliable change scores and high non-spurious correlations between change scores and independent criterion scores, even when pretest scores are not good predictors of either changes or the criterion…
Comparative Validity of Simple and Residualized Difference Scores
Apart from some anomalous exceptions which are not likely to occur in testing practice, the predictor-criterion validity of simple difference scores is greater than, less than, or equal to that of residualized difference scores depending on whether the correlation between observed pretest and posttest scores is greater than, less than, or equal to the ratio of the standard deviations of pretest and posttest scores
Element of Chance and Comparative Reliability of Matching Tests and Multiple-Choice Tests
Simple formulas for the reliability of matching tests and multiple-choice tests, based on the assumption that chance success by guessing is the only source of error variation, are derived. The reliabilities of the two types of tests are compared, and an explicit formula relating their reliability coefficients when relevant test parameters remain fixed is presented. It is found that, in the majority of cases which are likely in practice, matching …
A Note on the Correlation of Gains and Initial Status
By writing the familiar equation for the correlation of gains and initial status in a form which reveals that the correlation does not depend on the standard deviations of pretest and posttest scores separately, but only on their ratio, it is possible to exhibit a one-parameter family of functions which throws new light on this relationship. The same family of functions provides information about the correlation between the true components of gai…
The Mathematical Definition of Test Validity
A definition of test validity as the ratio of a covariance term to a variance term, analogous to the classical definition of test reliability as the proportion of observed variance which is true variance, is proposed. This definition leads to simple validity formulas which do not depend on an assumption of experimental independence. When error scores on distinct tests are uncorrelated, the proposed definition coincides with the usual one, but it …
Elimination of Bias in Test Scores
This paper demonstrates that correlation between bias components of test scores and unbiased observed scores is an effective predictor of changes in reliability and validity resulting from elimination of bias. Plausible assumptions about the value of this correlation and the sizes of related variance components yield the conclusion that reduction in reliability and validity is a possible outcome. This reduction could result in marginally valid or…
Mathematics (40 works) · Statistics (40 works) · Econometrics (23 works) · Psychology (22 works) · Advanced Statistical Methods and Models (20 works) · Test (biology (12 works) · Computer Science (11 works) · Psychometric Methodologies and Testing (11 works) · Reliability (semiconductor) (11 works) · Statistical hypothesis testing (11 works)