Fifty years later and still working
Rediscovering Paulus et al’s (1970) automated scoring of divergent thinking tests
Bibliographic Data
| ID | 21832579 |
|---|---|
| Authors | Boris Forthmann (0000-0001-9755-7304, University of Münster, corresponding author), Philipp Doebler (0000-0002-2946-8526, TU Dortmund University) |
| Year | 2025 |
| Volume | 19 |
| Issue | 1 |
| Pages | 63-76 |
| Publication date | 2025-02-01 |
| Peer Reviewed | Yes |
| Open Access | No |
| Type | ARTICLE |
| Venue | Psychology of Aesthetics Creativity and the Arts (JOURNAL) |
| Journal identifiers | ISSN: 1931-3896 • E-ISSN: 1931-390X |
| Publisher | American Psychological Association (APA) (PUBLISHER) |
| DOI | 10.1037/aca0000518 |
| OpenAlex | W4297999143 |
| Language | EN |
| Citations received | 7 |
| References cited | 20 |
Automated scoring of divergent thinking tasks is a current hot topic in creativity research.Most of the debated approaches are unsupervised machine learning approaches and researchers seemingly just started to evaluate supervised approaches.Hence, rediscovering the seminal work of Paulus et al. (1970) came as a big surprise to us.More than fifty years ago, they derived prediction formulas for an automated scoring of the Torrance Test of Creative Thinking that was based on a set of text mining variables (e.g., average word length, word counts, and so forth).They found quite impressive cross-validation results.This work reintroduces Paulus et al.'s (1970) approach and investigates how it performs compared to semantic distance scoring.The main contribution of Paulus et al.'s (1970) neglected masterpiece on divergent thinking assessment is echoed by the findings of this work: Creative quality of responses can be well predicted by means of simple text mining statistics.The validity was also stronger as compared to semantic distance.Importantly, using the Paulus et al. (1970) features in a state-of-the-art supervised machine learning approach does not outperform the simple stepwise regression used by Paulus et al. (1970).Yet, we found that supervised machine learning can outperform the Paulus et al. (1970) approach, when semantic distance is added to the set of prediction variables.We discuss challenges that are expected for future research that aim at combining unsupervised approaches based on word embeddings and supervised learning relying on text mining features
“Robust” but Tiny
Validating LLM-based alternative uses test scoring across ages
Automated Scoring of Question Complexity with Transformer Language Models
Investigating the Validity Evidence of Automated Scoring Methods for Divergent Thinking Assessments
Automatic Scoring of Metaphor Creativity with Large Language Models
What Makes Children's Responses to Creativity Assessments Difficult to Judge Reliably
Automated Scoring of Figural Tests of Creativity with Computer Vision
The nature of human intelligence
Development and Psychometric Integrity of a Measure of Ideational Behavior
Investigating the performance of exploratory graph analysis and traditional techniques to identify the number of latent factors
Explaining human performance in psycholinguistic tasks with models of semantic similarity based on prediction and counting
Scoring divergent thinking tests
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
A solution to Plato's problem
The WaCky wide web
Assessing creativity with divergent thinking tasks
Automating creativity assessment with SemDis
Greedy function approximation
Measuring divergent thinking originality with human raters and text-mining models
Understanding the confounding effect of fluency in divergent thinking scores
An introduction to latent semantic analysis
Scrutinizing the basis of originality in divergent thinking tests
SimLex-999
Intraclass correlations
| Unique citing works | 7 |
|---|---|
| Citations per year | 2,33 |
| Citation span | 2023 - 2026 (4) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 7 |