Ronald K Hambleton
Biographic Data
| ID | 265839 |
|---|---|
| NAME | Ronald K Hambleton |
| GIVEN NAMES | Ronald K |
| FAMILY NAME | Hambleton |
| SIGNATURE | HAMBLETON R K |
| AFFILIATIONS | University of Massachusetts Amherst |
| ORCID | 0000-0002-7412-2188 |
| VERIFIED | No |
| TOTAL WORKS | 31 |
| TOTAL CITATIONS | 22 |
| WORKS AS AUTHOR | 29 |
| WORKS AS EDITOR | 2 |
| FIRST PUBLICATION YEAR | 1969 |
| LATEST PUBLICATION YEAR | 2017 |
| H-INDEX | 2 |
The Effect of Rating Unfamiliar Items on Angoff Passing Scores
The Angoff standard setting method relies on content experts to review exam items and make judgments about the performance of the minimally proficient examinee. Unfortunately, at times content experts may have gaps in their understanding of specific exam content. These gaps are particularly likely to occur when the content domain is broad and/or highly technical, or when non-expert stakeholders are included in a standard setting panel (e.g., pare…
Comparative Analyses of Mirt Models and Software (BMIRT and flexMirt)
Application of MIRT modeling procedures is dependent on the quality of parameter estimates provided by the estimation software and techniques used. This study investigated model parameter recovery of two popular MIRT packages, BMIRT and flexMIRT, under some common measurement conditions. These packages were specifically selected to investigate the model parameter recovery of three item parameter estimation techniques, namely, Bock–Aitkin EM (BA-E…
An application of item response theory to psychological test development
Item response theory (IRT) has become a popular methodological framework for modeling response data from assessments in education and health; however, its use is not widespread among psychologists. This paper aims to provide a didactic application of IRT and to highlight some of these advantages for psychological test development. IRT was applied to two scales (a positive and a negative affect scale) of a self-report test. Respondents were 853 un…
Directrices para la traducción y adaptación de los tests
BACKGROUND: Adapting tests across cultures is a common practice that has increased in all evaluation areas in recent years. We live in an increasingly multicultural and multilingual world in which the tests are used to support decision-making in the educational, clinical, organizational and other areas, so the adaptation of tests becomes a necessity. The main goal of this paper is to present the second edition of the guidelines of the Internation…
Measuring global physical health in children with cerebral palsy
Psychometric Evaluation and Calibration of Health-Related Quality of Life Item Banks
BACKGROUND: The construction and evaluation of item banks to measure unidimensional constructs of health-related quality of life (HRQOL) is a fundamental objective of the Patient-Reported Outcomes Measurement Information System (PROMIS) project. OBJECTIVES: Item banks will be used as the foundation for developing short-form instruments and enabling computerized adaptive testing. The PROMIS Steering Committee selected 5 HRQOL domains for initial f…
Good Practices for Identifying Differential Item Functioning
S182-S188
Adapting Educational and Psychological Tests for Cross-Cultural Assessment
Contents: Preface. Part I: Cross-Cultural Adaptation of Educational and Psychological Tests: Theoretical and Methodological Issues. R.K. Hambleton, Issues, Designs, and Technical Guidelines for Adapting Tests Into Multiple Languages and Cultures. F.J.R. van de Vijver, Y.H. Poortinga, Conceptual and Methodological Issues in Adapting Tests. T. Oakland, Selected Ethical Issues Relevant to Test Adaptations. S.G. Sireci, L. Patsula, R.K. Hambleton, St…
Impact of Test Design, Item Quality, and Item Bank Size on the Psychometric Properties of Computer-Based Credentialing Examinations
Computer-based testing by credentialing agencies has become common; however, selecting a test design is difficult because several good ones are available—parallel forms, computer adaptive (CAT), and multistage (MST). In this study, three computerbased test designs under some common examination conditions were investigated. Item bank size and item quality had a practically significant impact on decision consistency and accuracy. Even in nearly ide…
Detection Of Differential Item Functioning In Large-Scale State Assessments
In differential item functioning (DIF) studies, examinees from different groups are typically ability matched, and then one or more statistical indices are used to compare performance on a set of test items. Typically, matching is on total test score (a criterion both observable and easily accessible), but it may be limited in value because if DIF is present, it is likely to distort test scores and potentially confound any item performance differ…
Advances in translating and adapting educational and psychological tests
The Next Generation of the ITC Test Translation and Adaptation Guidelines
The ITC Test Translation and Adaptation Guidelines have been available for nearly 7 years, and numerous researchers and practitioners have provided comments on their strengths and weaknesses. This paper addresses three goals: First, comments on the 22 guidelines that have resulted from the numerous field-tests and reviews are presented. Second, where possible, specific suggestions for revising the ITC Guidelines are described. Finally, three sugg…
Response to Hays et al and McHorney and Cohen
In summary, readers are encouraged to read carefully the IRT articles in this special issue. They provide theoretical as well as practical arguments in favor of using IRT models in the health outcomes measurement field. At the same time, readers should be aware that the IRT field is complex and software is limited and, in my judgment, not very user friendly (although some packages are better than others). In addition, sample sizes will often need…
Adapting Tests for Use in Multiple Languages and Cultures
Handbook of Modern Item Response Theory
A New Standard-Setting Method for Performance Assessments
Traditional standard-setting methods are not well suited for applications with polytomously scored performance assessments. The present article presents a standard-setting method-the dominant profile judgment (DPJ) method-designed for use with profiles of polytomous scores on exercises in a performance-based assessment. The method is direct, in that it guides standard-setfing panelists in the articulation of their standard-setting policies. Furth…
Translating Tests
With the increasing interest in cross-cultural research, there is a growing need for standard and validated practices for translating psychological instruments. Developing a psychologically acceptable instrument for another cultural group almost always requires more effort than a literal translation, which all too often is the common practice. The adequacy of translations can be threatened by various sources of bias. Three types of bias are disti…
Development of the Multidimensional Index of Life Quality
OBJECTIVES: The purpose of this study was to design a multidimensional measure of health-related quality of life appropriate for patients with cardiovascular disease that was psychometrically sound, brief, and easy to administer. METHODS: Qualitative interviews conducted with healthy subjects and patients with cardiovascular diseases identified nine major quality of life domains. Based on the responses of 129 cardiovascular disease patients recru…
Identification of Nonuniform Differential Item Functioning Using a Variation of the Mantel-Haenszel Procedure
The Mantel-Haenszel (MH) procedure has become one of the most popular procedures for detecting differential item functioning (DIF). One of the most troublesome criticisms of this procedure is that whereas detection rates for uniform DIF are very good, the procedure is not sensitive to nonuniform DIF. In this study, examinee responses were generated to simulate both uniform and nonuniform DIF. A standard MH procedure was used first. Then, examinee…
Mh
This software program computes the Mantel-Haenszel statistic that is used for detecting differential item functioning (DIF). The main features of the program are (a) the capability of using total test score or a "purified" test score as the internal criterion for matching examinees, (b) using an external criterion for matching examinees if one is available, (c) complete descriptive statistics on the test data, (d) complete reference and focal gro…
An NCME Instructional Module on
The Effect of Sample Size on the Functioning of the Mantel-Haenszel Statistic
The Mantel-Haenszel (MH) procedure has become one of the most popular procedures for detecting differential item functioning. Valid results with relatively small numbers of examinees are one of the advantages typically attributed to this procedure. In this study, examinee item responses were simulated to contain differentially functioning items, and then were analyzed at five sample sizes to compare detection rates. Results showed the MH procedur…
Fundamentals of Item Response Theory
Evaluation of Computer Simulated Baseline Statistics for Use in Item Bias Studies
Though item bias statistics are widely recommended for use in test development and test analysis work, problems arise in their interpretation. The purpose of the present research was to evaluate the validity of logistic test models and computer simulation methods for providing a frame of reference for item bias statistic interpretations. Specifically, the intent was to produce simulated sampling distributions of item bias statistics under the hyp…
Determining Optimal Test Lengths with a Fixed Total Testing Time
The problem of determining optimal test lengths with fixed total testing time has proven to be a difficult one for criterion-referenced test developers. A solution is offered in this paper in which test lengths are determined by considering the importance, reliability, and validity of scores measuring objectives in a criterion-referenced test with a constraint set on the total desired test length
Adapting Tests for Use in Multiple Languages and Cultures
An application of item response theory to psychological test development
Item response theory (IRT) has become a popular methodological framework for modeling response data from assessments in education and health; however, its use is not widespread among psychologists. This paper aims to provide a didactic application of IRT and to highlight some of these advantages for psychological test development. IRT was applied to two scales (a positive and a negative affect scale) of a self-report test. Respondents were 853 un…
Using the Sentence Verification Technique To Assess the Comprehension of Technical Text as a Function of Subject Matter Expertise
We report four studies that are concerned with the construct validity of the Sentence Verification Technique (SVT) as a measure of reading comprehension. The studies proceed from the assumption that readers with greater subject matter expertise should comprehend technical text in their area of expertise better than readers with less subject matter expertise. Experiment 1 had graduate students in psychology, advanced undergraduate psychology major…
Effects of Promised Reward and Threatened Penalty on Performance of a Multiple-Choice Vocabulary Test
The Effect of Scoring Instructions and Degree of Speededness on the Validity and Reliability of Multiple-Choice Tests1
A General Fortran IV Test Scoring Program
A Computer Simulation Study of Tailored Testing Strategies for Objective-Based Instructional Programs
One possible way of reducing the amount of time spent testing in . objective-based instructional programs would involve the implementation of a tailored testing strategy. Our purpose was to provide some additional data on the effectiveness of various tailored testing strategies for different testing situations. The three factors of a tailored testing strategy under study with various hypothetical distributions of abilities across two learning hie…
Using the Sentence Verification Technique To Assess the Comprehension of Technical Text as a Function of Subject Matter Expertise
We report four studies that are concerned with the construct validity of the Sentence Verification Technique (SVT) as a measure of reading comprehension. The studies proceed from the assumption that readers with greater subject matter expertise should comprehend technical text in their area of expertise better than readers with less subject matter expertise. Experiment 1 had graduate students in psychology, advanced undergraduate psychology major…
Item Response Theory
Determining Optimal Test Lengths with a Fixed Total Testing Time
The problem of determining optimal test lengths with fixed total testing time has proven to be a difficult one for criterion-referenced test developers. A solution is offered in this paper in which test lengths are determined by considering the importance, reliability, and validity of scores measuring objectives in a criterion-referenced test with a constraint set on the total desired test length
Evaluation of Computer Simulated Baseline Statistics for Use in Item Bias Studies
Though item bias statistics are widely recommended for use in test development and test analysis work, problems arise in their interpretation. The purpose of the present research was to evaluate the validity of logistic test models and computer simulation methods for providing a frame of reference for item bias statistic interpretations. Specifically, the intent was to produce simulated sampling distributions of item bias statistics under the hyp…
The Effect of Sample Size on the Functioning of the Mantel-Haenszel Statistic
The Mantel-Haenszel (MH) procedure has become one of the most popular procedures for detecting differential item functioning. Valid results with relatively small numbers of examinees are one of the advantages typically attributed to this procedure. In this study, examinee item responses were simulated to contain differentially functioning items, and then were analyzed at five sample sizes to compare detection rates. Results showed the MH procedur…
Fundamentals of Item Response Theory
An NCME Instructional Module on
Identification of Nonuniform Differential Item Functioning Using a Variation of the Mantel-Haenszel Procedure
The Mantel-Haenszel (MH) procedure has become one of the most popular procedures for detecting differential item functioning (DIF). One of the most troublesome criticisms of this procedure is that whereas detection rates for uniform DIF are very good, the procedure is not sensitive to nonuniform DIF. In this study, examinee responses were generated to simulate both uniform and nonuniform DIF. A standard MH procedure was used first. Then, examinee…
Mh
This software program computes the Mantel-Haenszel statistic that is used for detecting differential item functioning (DIF). The main features of the program are (a) the capability of using total test score or a "purified" test score as the internal criterion for matching examinees, (b) using an external criterion for matching examinees if one is available, (c) complete descriptive statistics on the test data, (d) complete reference and focal gro…
Translating Tests
With the increasing interest in cross-cultural research, there is a growing need for standard and validated practices for translating psychological instruments. Developing a psychologically acceptable instrument for another cultural group almost always requires more effort than a literal translation, which all too often is the common practice. The adequacy of translations can be threatened by various sources of bias. Three types of bias are disti…
Development of the Multidimensional Index of Life Quality
OBJECTIVES: The purpose of this study was to design a multidimensional measure of health-related quality of life appropriate for patients with cardiovascular disease that was psychometrically sound, brief, and easy to administer. METHODS: Qualitative interviews conducted with healthy subjects and patients with cardiovascular diseases identified nine major quality of life domains. Based on the responses of 129 cardiovascular disease patients recru…
Handbook of Modern Item Response Theory
A New Standard-Setting Method for Performance Assessments
Traditional standard-setting methods are not well suited for applications with polytomously scored performance assessments. The present article presents a standard-setting method-the dominant profile judgment (DPJ) method-designed for use with profiles of polytomous scores on exercises in a performance-based assessment. The method is direct, in that it guides standard-setfing panelists in the articulation of their standard-setting policies. Furth…
Adapting Tests for Use in Multiple Languages and Cultures
Response to Hays et al and McHorney and Cohen
In summary, readers are encouraged to read carefully the IRT articles in this special issue. They provide theoretical as well as practical arguments in favor of using IRT models in the health outcomes measurement field. At the same time, readers should be aware that the IRT field is complex and software is limited and, in my judgment, not very user friendly (although some packages are better than others). In addition, sample sizes will often need…
The Next Generation of the ITC Test Translation and Adaptation Guidelines
The ITC Test Translation and Adaptation Guidelines have been available for nearly 7 years, and numerous researchers and practitioners have provided comments on their strengths and weaknesses. This paper addresses three goals: First, comments on the 22 guidelines that have resulted from the numerous field-tests and reviews are presented. Second, where possible, specific suggestions for revising the ITC Guidelines are described. Finally, three sugg…
Detection Of Differential Item Functioning In Large-Scale State Assessments
In differential item functioning (DIF) studies, examinees from different groups are typically ability matched, and then one or more statistical indices are used to compare performance on a set of test items. Typically, matching is on total test score (a criterion both observable and easily accessible), but it may be limited in value because if DIF is present, it is likely to distort test scores and potentially confound any item performance differ…
Advances in translating and adapting educational and psychological tests
Adapting Educational and Psychological Tests for Cross-Cultural Assessment
Contents: Preface. Part I: Cross-Cultural Adaptation of Educational and Psychological Tests: Theoretical and Methodological Issues. R.K. Hambleton, Issues, Designs, and Technical Guidelines for Adapting Tests Into Multiple Languages and Cultures. F.J.R. van de Vijver, Y.H. Poortinga, Conceptual and Methodological Issues in Adapting Tests. T. Oakland, Selected Ethical Issues Relevant to Test Adaptations. S.G. Sireci, L. Patsula, R.K. Hambleton, St…
Impact of Test Design, Item Quality, and Item Bank Size on the Psychometric Properties of Computer-Based Credentialing Examinations
Computer-based testing by credentialing agencies has become common; however, selecting a test design is difficult because several good ones are available—parallel forms, computer adaptive (CAT), and multistage (MST). In this study, three computerbased test designs under some common examination conditions were investigated. Item bank size and item quality had a practically significant impact on decision consistency and accuracy. Even in nearly ide…
Good Practices for Identifying Differential Item Functioning
S182-S188
Psychology (26 works) · Psychometric Methodologies and Testing (20 works) · Psychometrics (20 works) · Computer Science (18 works) · Mathematics (17 works) · Statistics (16 works) · Item response theory (14 works) · Test (biology) (13 works) · Clinical Psychology (9 works) · Differential item functioning (7 works)