The problem of reliability in the study of early communication skills

The question of the reliability of data in studies of child language has rarely been addressed although the objectivity of such observation is open to question. The social and communicative behaviour of 28 preschool children was classified in terms of pragmatic functions (for gestures and vocalizations) or semantic functions (for single words). Nineteen of these children were developmentally delayed. Each child was observed for 15 min sessions by each of four observers. Only 13 of the 40 behavioural codes were stable across these four observation sessions. Fifteen of the children were jointly observed on 27 occasions yielding 83 inter‐observer agreement comparisons among the four observers. Cohen's kappa (a measure of percentage agreement after chance agreement has been removed) ranged from 0.01 to 0.66 (mean = 0.29, SD = 0.22). Pearson correlations for the total frequency of each behaviour by each observer during a 15 min observation ranged from 0 03 to 0.99 (mean = 0.84, SD = 0.15). Four children were videotaped and transcripts of their behaviour were independently rated by two observers producing an average kappa of 0.74. Finally inter‐observer agreement from observations of developmentally delayed and normal children was compared. No statistically significant difference was found. The results were discussed in terms of the necessity for measures of reliability, the appropriate level of data analysis in light of such measures, and the problem of inter‐subjectivity in the study of early communicative development

Audiology · Developmental psychology · Gesture · Kappa · Language development · Linguistics · Observer (physics) · Reliability (semiconductor) · Early Childhood Education and Development · Infant Development and Preterm Care · Language Development and Disorders · Psychology