Gary J Ockey
Biographic Data
| ID | 5997659 |
|---|---|
| NAME | Gary J Ockey |
| GIVEN NAMES | Gary J |
| FAMILY NAME | Ockey |
| SIGNATURE | OCKEY G J |
| AFFILIATIONS | Iowa State University |
| ORCID | 0000-0002-7855-1676 |
| VERIFIED | Yes |
| TOTAL WORKS | 14 |
| TOTAL CITATIONS | 2 |
| AUTHOR COUNT | 14 |
| EDITOR COUNT | 0 |
| FIRST PUBLICATION YEAR | 2003 |
| LATEST PUBLICATION YEAR | 2024 |
| H-INDEX | 1 |
Open Science in language assessment research contexts: A reply to Winke
Investigation into the factor structure of a local English placement test of oral communication
This study describes the development process and examines the construct validity of an English placement test of oral communication (EPT OC) developed at a Midwestern university in the United States. This test includes a one‐on‐one oral interview and paired discussion task, and test performance is judged on an analytic rating scale. A confirmatory factor analysis conducted on the ratings of 338 students who took the initial fully operational EPT …
Human versus Computer Partner in the Paired Oral Discussion Test
A challenge of large-scale oral communication assessments is to feasibly assess a broad construct that includes interactional competence. One possible approach in addressing this challenge is to use a spoken dialog system (SDS), with the computer acting as a peer to elicit a ratable speech sample. With this aim, an SDS was built and four trained human raters assessed the discourse elicited from 40 test takers that completed a paired oral task wit…
Establishing Appropriate Cut Scores of a Standardized Test for a Local Placement Context
Various approaches help determine students’ need for language support courses. Some programs use standardized language proficiency test scores, which are available from the admissions process; others use local placement tests, which are typically designed to be aligned with local language needs and ESL course curricula. Using local placement tests may be more defensible, but sufficient resources to develop, administer, and score valid assessments…
Assessment of Listening
Valid assessment of listening comes with many challenges and unresolved issues. A variety of types of input and tasks for assessing comprehension have been used, all of which have strengths and limitations. The quality of listening assessments should be evaluated based on the purpose of the test, the characteristics of the test takers, and the context in which the assessment will be given. Developing computer technology continues to increase the …
New and not so new methods for assessing oral communication
Author/s Gary J. OckeyZhi LiIowa State University, USA ABSTRACT The assessment of oral communication has continued to evolve over the past few decades. The construct being assessed has broadened to include interactional competence, and technology has played a role in the types of tasks that are currently popular. In this paper, we discuss the factors that affect the process of oral communication assessment, current conceptualizations of the const…
Item Response Theory
Item response theory (IRT) is an approach which can be used to estimate how much ability a person possesses on an unobservable construct, including those of interest to applied linguists. The theory assumes that a person can be placed on an ability continuum based on a score assigned to an observable performance.
From One to Multiple Accents on a Test of L2 Listening Comprehension
Concerns about the need for assessing multidialectal listening skills for global contexts are becoming increasingly prevalent. However, the inclusion of multiple accents on listening assessments may threaten test fairness because it is not practical to include every accent that may be encountered in the language use domain on these tests. Given this dilemma, this study aimed to determine the extent to which accent strength and familiarity affect …
The extent to which Toefl iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students
The purpose of this study was to determine the extent to which performance on the TOEFL iBT speaking section is associated with other indicators of Japanese university students’ abilities to communicate orally in an academic English environment and to determine which components of oral ability for these tasks are best assessed by TOEFL iBT. To achieve this aim, TOEFL iBT speaking scores were compared to performances on a group oral discussion, pi…
Assessment of Listening
Listening is important for in‐person communication with estimates that it accounts for more than 45% of the time spent communicating (Feyten, 1991). Listening continues to become more important in virtual environments with the increase of communication through such technologies as video broadcasting and Skype. It follows that teaching and assessing the listening skill of second language learners is essential. Unfortunately, the assessment of seco…
Self‐consciousness and Assertiveness as Explanatory Variables of L2 Oral Ability: A Latent Variable Approach
Drawing on current theories in personality, second‐language (L2) oral ability, and psychometrics, this study investigates the extent to which self‐consciousness and assertiveness are explanatory variables of L2 oral ability. Three hundred sixty first‐year Japanese university students who were studying English as a foreign language participated in the study. Personality was measured with the Japanese version of the NEO‐PI‐R ( Shimonaka, Nakazato, …
The effects of group members' personalities on a test taker's L2 group oral discussion test scores
The second language group oral is a test of second language speaking proficiency, in which a group of three or more English language learners discuss an assigned topic without interaction with interlocutors. Concerns expressed about the extent to which test takers' personal characteristics affect the scores of others in the group have limited its attractiveness. This study investigates the degree to which assertive and non-assertive test takers' …
Construct implications of including still image or video in computer-based listening tests
Over the past decade, listening comprehension tests have been converting to computer-based tests that include visual input. However, little research is available to suggest how test takers engage with different types of visuals on such tests. The present study compared a series of still images to video in academic computer-based tests to determine how test takers engage with these two test modes. The study, which employed observations, retrospect…
A many-facet Rasch analysis of the second language group oral discussion task
FACETS many-facet Rasch analysis software (Linacre, 1998a) was utilized to look at two consecutive administrations of a large-scale (more than 1000 examinees) second language oral assessment in the form of a peer group discussion task with Japanese English-major university students. Facets modeled in the analysis were examinee, prompt, rater, and five rating category ‘items.’ Unidimensionality was shown to be strong in both datasets, and approach…
Self‐consciousness and Assertiveness as Explanatory Variables of L2 Oral Ability: A Latent Variable Approach
Drawing on current theories in personality, second‐language (L2) oral ability, and psychometrics, this study investigates the extent to which self‐consciousness and assertiveness are explanatory variables of L2 oral ability. Three hundred sixty first‐year Japanese university students who were studying English as a foreign language participated in the study. Personality was measured with the Japanese version of the NEO‐PI‐R ( Shimonaka, Nakazato, …
A many-facet Rasch analysis of the second language group oral discussion task
FACETS many-facet Rasch analysis software (Linacre, 1998a) was utilized to look at two consecutive administrations of a large-scale (more than 1000 examinees) second language oral assessment in the form of a peer group discussion task with Japanese English-major university students. Facets modeled in the analysis were examinee, prompt, rater, and five rating category ‘items.’ Unidimensionality was shown to be strong in both datasets, and approach…
Construct implications of including still image or video in computer-based listening tests
Over the past decade, listening comprehension tests have been converting to computer-based tests that include visual input. However, little research is available to suggest how test takers engage with different types of visuals on such tests. The present study compared a series of still images to video in academic computer-based tests to determine how test takers engage with these two test modes. The study, which employed observations, retrospect…
The effects of group members' personalities on a test taker's L2 group oral discussion test scores
The second language group oral is a test of second language speaking proficiency, in which a group of three or more English language learners discuss an assigned topic without interaction with interlocutors. Concerns expressed about the extent to which test takers' personal characteristics affect the scores of others in the group have limited its attractiveness. This study investigates the degree to which assertive and non-assertive test takers' …
Self‐consciousness and Assertiveness as Explanatory Variables of L2 Oral Ability: A Latent Variable Approach
Drawing on current theories in personality, second‐language (L2) oral ability, and psychometrics, this study investigates the extent to which self‐consciousness and assertiveness are explanatory variables of L2 oral ability. Three hundred sixty first‐year Japanese university students who were studying English as a foreign language participated in the study. Personality was measured with the Japanese version of the NEO‐PI‐R ( Shimonaka, Nakazato, …
Assessment of Listening
Listening is important for in‐person communication with estimates that it accounts for more than 45% of the time spent communicating (Feyten, 1991). Listening continues to become more important in virtual environments with the increase of communication through such technologies as video broadcasting and Skype. It follows that teaching and assessing the listening skill of second language learners is essential. Unfortunately, the assessment of seco…
The extent to which Toefl iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students
The purpose of this study was to determine the extent to which performance on the TOEFL iBT speaking section is associated with other indicators of Japanese university students’ abilities to communicate orally in an academic English environment and to determine which components of oral ability for these tasks are best assessed by TOEFL iBT. To achieve this aim, TOEFL iBT speaking scores were compared to performances on a group oral discussion, pi…
From One to Multiple Accents on a Test of L2 Listening Comprehension
Concerns about the need for assessing multidialectal listening skills for global contexts are becoming increasingly prevalent. However, the inclusion of multiple accents on listening assessments may threaten test fairness because it is not practical to include every accent that may be encountered in the language use domain on these tests. Given this dilemma, this study aimed to determine the extent to which accent strength and familiarity affect …
Item Response Theory
Item response theory (IRT) is an approach which can be used to estimate how much ability a person possesses on an unobservable construct, including those of interest to applied linguists. The theory assumes that a person can be placed on an ability continuum based on a score assigned to an observable performance.
New and not so new methods for assessing oral communication
Author/s Gary J. OckeyZhi LiIowa State University, USA ABSTRACT The assessment of oral communication has continued to evolve over the past few decades. The construct being assessed has broadened to include interactional competence, and technology has played a role in the types of tasks that are currently popular. In this paper, we discuss the factors that affect the process of oral communication assessment, current conceptualizations of the const…
Assessment of Listening
Valid assessment of listening comes with many challenges and unresolved issues. A variety of types of input and tasks for assessing comprehension have been used, all of which have strengths and limitations. The quality of listening assessments should be evaluated based on the purpose of the test, the characteristics of the test takers, and the context in which the assessment will be given. Developing computer technology continues to increase the …
Human versus Computer Partner in the Paired Oral Discussion Test
A challenge of large-scale oral communication assessments is to feasibly assess a broad construct that includes interactional competence. One possible approach in addressing this challenge is to use a spoken dialog system (SDS), with the computer acting as a peer to elicit a ratable speech sample. With this aim, an SDS was built and four trained human raters assessed the discourse elicited from 40 test takers that completed a paired oral task wit…
Establishing Appropriate Cut Scores of a Standardized Test for a Local Placement Context
Various approaches help determine students’ need for language support courses. Some programs use standardized language proficiency test scores, which are available from the admissions process; others use local placement tests, which are typically designed to be aligned with local language needs and ESL course curricula. Using local placement tests may be more defensible, but sufficient resources to develop, administer, and score valid assessments…
Investigation into the factor structure of a local English placement test of oral communication
This study describes the development process and examines the construct validity of an English placement test of oral communication (EPT OC) developed at a Midwestern university in the United States. This test includes a one‐on‐one oral interview and paired discussion task, and test performance is judged on an analytic rating scale. A confirmatory factor analysis conducted on the ratings of 338 students who took the initial fully operational EPT …
Open Science in language assessment research contexts: A reply to Winke
Psychology (14 works) · EFL/ESL Teaching and Learning (11 works) · Computer Science (8 works) · Developmental psychology (7 works) · Linguistics (7 works) · Mathematics education (7 works) · Social Psychology (7 works) · Pronunciation (5 works) · Psychometrics (5 works) · Active listening (4 works)